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ABSTRACT 


The  Department  of  Defense  (DoD)  is  interested  in  acquiring  systems  that  promote  the  use 
of  open  architecture  (OA).  Industry  has  successfully  implemented  service-oriented 
architecture  (SOA)  in  its  processes  and  may  provide  a  benchmark  for  cost  savings  as  well 
as  examples  of  best  practices  for  the  DoD.  The  basic  research  question  guiding  this  thesis 
is,  What  are  the  industry  cost-saving  benchmarks  when  transitioning  to  SOA  from  a 
proprietary  system?  The  research  supports  the  argument  that  OA  in  the  DoD  is  similar  to 
SOA  in  industry.  This  comparison  is  essential  for  the  application  of  this  thesis  because 
this  allows  the  outcomes  of  industry  SOA  implementation  to  be  translated  into  what  the 
DoD  can  expect  from  its  OA  implementations.  This  research  then  answers  the  research 
question  by  analyzing  34  industry  reports,  18  of  which  provided  at  least  an  overall  ROI, 
and  10  of  which  broke  out  their  ROI  calculations  into  separate  cost  types.  The  reported 
costs  were  grouped  into  categories  of  cost  savings,  cost  avoidance,  or  productivity 
improvements.  The  researcher  concluded  that  the  industry  ROI  for  SOA  implementation 
is  72%.  Additionally,  best  practices  in  industry  that  are  transferable  to  DoD  were 
indentified,  including  ensuring  system  flexibility  and  implementing  SOA  incrementally. 
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I.  INTRODUCTION 


A.  PURPOSE 

The  purpose  of  this  thesis  is  to  analyze  cost  savings  from  various  industries 
following  the  implementation  of  a  service-oriented  architecture  (SOA).  In  order  to 
accomplish  this,  the  researcher  analyzed  cost  implications  that  result  from  industries 
moving  from  a  proprietary  architecture  to  an  SOA.  The  objective  of  this  thesis  is  to 
establish  a  benchmark  of  perfonnance  outcomes,  focusing  on  cost  savings  that  were 
experienced  in  industry  in  order  to  determine  what  the  government,  or  the  Department  of 
Defense  (DoD)  specifically,  can  expect  to  realize  in  its  push  to  move  to  a  more  open 
architecture  model.  In  addition,  the  researcher  determined  some  industry  best  practices 
that  may  be  used  by  the  DoD  as  it  moves  to  an  open  architecture  model. 

B.  BACKGROUND 

Traditionally,  the  Navy  has  had  rather  inflexible  acquisition  strategies  and  has 
locked  itself  into  single  -stove-piped”  systems  that  typically  perfonn  well  but  tend  to  be 
localized  and  prevent  the  sharing  of  information  across  different  systems.  In  addition, 
because  the  Navy  is  locked  into  specific  systems,  the  options  of  vendors  who  supply 
these  systems  become  limited.  In  turn,  there  is  little  competition  to  drive  down  prices. 
The  results  are  systems  that  have  duplicative  capabilities  and  are  incompatible  with  other 
systems.  Each  system  has  become  unique  to  the  platform  for  which  it  was  originally 
designed. 

To  combat  this,  in  recent  years  the  Navy  has  promoted  the  use  of  open 
architecture  (OA)  in  acquisitions  as  a  way  to  field  systems  faster  and  at  a  lower  cost. 
Some  of  the  systems  that  adopted  OA  early  on  are  now  being  analyzed  to  detennine 
whether  they  are  achieving  the  promised  benefits.  However,  there  is  no  identified 
benchmark  by  which  to  compare  the  results  of  the  analyses.  By  looking  at  private 
industry  perfonnance  to  identify  a  benchmark,  the  Navy  can  better  determine  the  type  of 
results  it  should  be  receiving  from  its  investments. 
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One  way  to  analyze  benefits  is  through  the  use  of  the  financial  metric  return  on 
investment  (ROI).  ROI  has  long  been  accepted  in  industry  as  a  way  to  measure  the 
success  of  an  investment,  but  has  more  recently  been  promoted  for  use  within  the  DoD. 
The  Clinger-Cohen  Act  (1996)  mandates  the  assessment  of  cost  benefits  for  information 
technology  investments.  In  addition,  the  Government  Accountability  Office‘s  (GAO) 
Assessing  Risks  and  Returns:  A  Guide  for  Evaluating  Federal  Agencies  ’  IT  Investment 
Decision-Making,  Version  1,  (1997)  requires  that  infonnation  technology  (IT) 
investments  apply  ROI  measures.  Finally,  DoD  Directive  81 15.01  (Assistant  Secretary  of 
Defense  for  Networks  and  Infonnation  Integration,  2005)  issued  in  October  2005 
mandates  the  use  of  perfonnance  metrics  based  on  outputs,  with  ROI  analysis  required 
for  all  current  and  planned  IT  investments.  By  analyzing  the  ROI  achieved  in  industry 
and  comparing  them  to  OA  in  the  DoD,  the  DoD  will  take  one  more  step  to  reaching  the 
goals  set  forth  in  the  documents  described  in  this  section. 

C.  RESEARCH  OBJECTIVES 

The  research  conducted  for  this  thesis  encompassed  several  objectives.  The  first 
objective  was  to  examine  the  relationships  between  OA  and  SOA.  The  second  objective 
was  to  make  a  connection  between  Navy  OA  to  industry  SOA.  This  was  necessary  in 
order  to  apply  the  ROI  achieved  in  industry  as  an  applicable  benchmark  achievable  in  the 
DoD.  The  third  research  objective  was  to  establish  a  cost-savings  benchmark  based  on 
industry  performance  between  the  traditional  proprietary  architecture  model  and  the 
SOA.  The  fourth  and  final  objective  was  to  determine  some  industry  best  practices  that 
would  work  for  the  DoD,  as  well  as  to  identify  some  potential  inhibitors  that  the  DoD 
will  need  to  carefully  examine. 
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D.  RESEARCH  QUESTIONS 


In  this  thesis,  the  researcher  attempts  to  provide  decision  makers  with  answers  to 
several  questions,  as  well  as  to  provide  recommendations  for  future  studies. 

1.  What  are  the  industry  cost-saving  benchmarks  when  transitioning  to  SOA 
from  a  proprietary  system? 

2.  What  are  some  industry  best  practices  that  may  be  used  by  the  DoD? 

E.  METHODOLOGY 

The  researcher  analyzed  industry  published  reports  focusing  on  the  benefits 
provided  by  an  SOA.  Typically,  these  benefits  were  in  the  fonn  of  an  achieved  ROI.  The 
achieved  benefits  were  broken  down  as  much  as  possible  by  the  researcher  to  discover 
what  percentage  of  the  benefits  was  achieved  due  to  cost  savings,  as  well  as  other 
benefits.  If  detailed  cost  savings  could  not  be  determined,  generalities  were  formed  from 
surveys  and  overall  ROI  industry  reports.  This  data  was  analyzed  and  used  to  fonn  an 
industry  average  of  typical  cost  savings  achieved  following  an  implementation  of  SOA. 
In  addition,  any  correlations  discovered  were  noted,  as  well  as  any  patterns  that  point  to 
methods  of  best  practice. 

F.  SCOPE 

The  scope  of  this  thesis  is  primarily  concerned  with  cost  savings  that  can  be 
achieved  by  SOA.  However,  other  benefits  also  provide  value,  and  these  were  analyzed 
as  well.  Some  examples  of  these  benefits  are  quicker  response  time,  decreased  error  rate, 
and  increased  revenue.  Some  of  these  measures  are  somewhat  subjective  and  difficult  to 
quantify  and  were  not  included  in  calculating  cost  savings.  Although  these  are  mentioned 
as  additional  benefits,  they  were  still  analyzed  because  they  contribute  to  the  overall 
benefit  of  SOA  implementation. 

G.  THESIS  ORGANIZATION 

This  thesis  is  organized  to  present  a  sequential  flow  of  infonnation,  ending  with 
conclusions  and  recommendations  of  the  research.  The  chapters  are  organized  in  the 
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following  manner:  In  Chapter  I,  the  researcher  provides  an  overview  of  the  thesis  with 
regard  to  purpose,  methodology,  and  scope.  In  addition,  researched  questions  and 
objectives  are  identified.  In  Chapter  II,  the  researcher  provides  a  background  for 
understanding  Navy  OA  and  SOA.  Chapter  III  bridges  the  concepts  from  Chapter  II. 
Differences  and  commonalities  are  analyzed  and  the  conclusion  is  drawn  that  the 
principles  observed  in  industry  SOA  are  applicable  to  Navy  OA.  In  this  chapter,  the 
researcher  provides  the  foundation  for  the  information  needed  to  complete  the  research 
and  draw  conclusions.  Chapter  IV  introduces  ROI  as  it  applies  to  IT  investments. 
Chapter  V  is  a  detailed  synopsis  of  the  research  conducted  as  well  as  the  findings  and 
analysis  of  the  data.  This  is  the  chapter  that  will  ultimately  answer  the  thesis  question. 
Chapter  VI  presents  conclusions,  shortcomings  of  the  study,  and  recommendations  for 
future  research. 
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II.  LITERATURE  REVIEW 


A.  OPEN  VS.  CLOSED  SYSTEMS 

There  are  generally  two  types  of  IT  systems,  open  systems  and  closed  systems. 
Closed  systems  are  characterized  by  closely  held,  privately  owned  standards,  protocols, 
languages,  and  data  formats  that  are  either  unavailable  to  outsiders  or  are  available  only 
at  a  very  high  license  fee  (Azani,  2001).  Closed  systems  typically  contain  proprietary 
software  designed  for  the  purpose  of  supporting  a  single  system.  When  proprietary 
systems  require  upgrades  or  maintenance,  their  unique  design  makes  upgrades  costly  and 
technically  difficult,  which  leads  to  increases  in  the  total  life-cycle  cost  of  the  system. 
Since  the  systems  are  developed  for  a  single  purpose,  interoperability  with  other  systems 
suffers.  Many  times,  additional  -middleware”  is  inserted  to  achieve  interoperability 
between  systems  (middleware  is  software  that  connects  two  disparate  and  closed  systems 
together  through  the  use  of  defined  interfaces).  This  adds  another  layer  to  the  system  and 
is  potentially  more  costly  to  implement  and  maintain.  However,  when  systems  use  the 
open  architecture  approach,  middleware  solves  the  interoperability  issue. 

The  goal  of  systems  is  to  have  them  perform  better  and  be  more  cost  efficient. 
Open  systems  can  accomplish  this  task.  In  closed  systems,  upgrades  that  would  provide 
greater  processing  capacity  cannot  be  completed  without  overhauling  the  current  systems. 
However,  in  an  open  system,  the  hardware  and  software  can  be  modularized,  making 
upgrades  more  efficient.  Open  systems  take  advantage  of  commercial  advances  by  using 
commercial  off-the-shelf  (COTS)  technologies  to  the  fullest  extent.  This  enables  the  most 
current  technology  to  be  used  and  allows  for  competition  within  industry  (Uchytil,  2006). 
Closed  systems  tie  the  system  owner  to  one  sole-source  contractor.  Table  1  provides  a 
comparison  of  some  of  the  aspects  of  open  versus  closed  systems. 
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Table  1.  Open  Systems  vs.  Closed  Systems 

(From  Azani,  2001) 


Closed  System  Characteristics 

Open  System  Characteristics 

Use  of  closely  held,  private  interfaces, 
languages,  data  fonnats,  and  protocols 
(government  or  vendor  unique  standards) 

Use  of  publicly  available  and  widely  used 
interfaces,  languages,  data  formats,  and 
protocols 

Critical  importance  is  given  to  unique 
design  and  implementation 

Critical  importance  is  given  to  interfaces 
management  and  widely  used  conventions 

Less  emphasis  on  modularity 

Heavy  emphasis  on  modularity 

Vendor  and  technology  dependency 

Vendor  and  technology  independence 

Minimization  of  the  number  of 
implementations 

Minimization  of  the  number  of  types  of 
interfaces 

Difficult  and  more  costly  integration 

High  degree  of  portability,  connectivity, 
interoperability,  and  scalability 

Use  of  sole-source  vendor 

Use  of  multiple  vendors 

Expansion  and  upgrading  usually  requires 
considerable  time,  money,  and  effort 

Easier,  quicker,  and  less  expensive 
expansion  and  upgrading 

Higher  total  ownership  cost 

Lower  total  ownership  cost 

Slower  and  more  costly  technology  to 
transfer 

Technology  transfer  is  faster  and  less 
costly 

Components,  interfaces,  standards,  and 
implementations  are  selected  sequentially 

Components,  interfaces,  standards,  and 
implementations  are  selected  interactively 

Systems  with  shorter  life  expectancy 

Systems  with  longer  life  expectancy 

Use  of  individual  company  preferences  to 
set  and  maintain  specifications 

Use  of  group  consensus  process  to 
maintain  interface  specifications 

Less  adaptable  to  change  in  threats  and 
technologies 

More  adaptable  to  evolving  threats  and 
technologies 

Focusing  mostly  on  development  cost  and 
meeting  present  mission 

Focusing  on  total  costs  of  ownership, 
sustainment,  and  growth 

User  as  the  producer  of  system 

User  as  the  consumer  of  components 

Rigid  and  slow  system  of  influence  and 
control 

Real  time  and  cybernetic  system  of 
influence  and  control 

Adversarial  relationship  with  prime 
contractors/supplier/vendors 

Symbiotic  relationship  with  prime 
contractors/suppliers/vendors 

Mostly  confined  to  traditional  suppliers 

Non- traditional  suppliers  can  compete 

Simple  confonnance  testing 

Very  challenging  confonnance  testing 

Many  current  legacy  systems  in  the  DoD  and  the  Navy,  in  particular,  follow  the 
closed,  proprietary  system.  The  Navy‘s  OA  model  was  implemented  to  move  the  Navy 
away  from  the  acquisition  of  closed  systems  to  field  open  systems. 
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B.  OPEN  ARCHITECTURE 

1.  Definition 

An  open  architecture  is  an  architecture  that  employs  open  standards  for  key 
interfaces  within  a  system  (Open  Systems  Joint  Task  Force  [OSJTF],  n.d.).  This  allows 
the  components  of  a  system  to  be  interchangeable  with  other  systems.  One  simple 
example  of  this  is  plug-and-play  computer  accessories.  OA  follows  principles  that  enable 
modular,  interoperable  systems  to  adhere  to  open  standards.  Open  standards  are  simply 
standards  that  are  widely  used,  consensus  based,  published,  and  maintained  by 
recognized  industry  standards  organizations  (OSJTF,  n.d.).  There  are  four  primary  types 
of  standards:  formal  standards,  industry  standards,  de  facto  standards,  and  proprietary 
standards.  Formal  standards  are  standards  that  are  formally  recognized  by  a  standards 
committee.  Industry  standards  are  fonnal  or  de  facto  standards  that  are  widely  accepted 
and  broadly  implemented.  De  facto  standards  are  standards  that  are  not  fonnal  standards, 
but  have  gained  widespread  acceptance  by  users.  Proprietary  standards  are  standards  that 
have  been  published  but  the  number  of  vendor  implementations  is  limited. 

The  goals  of  OA  are  to  increase  reuse,  increase  flexibility,  shorten  delivery  time 
to  market,  reduce  costs,  leverage  competition,  and  improve  interoperability.  Of  these,  the 
key  reasons  the  Navy  is  interested  in  OA  are  the  decreased  delivery  time  and  the  assumed 
reduction  in  total  ownership  costs. 

As  OA  was  gaining  hold  in  the  commercial  sector,  the  Navy  wanted  to  take 
advantage  of  the  benefits  that  OA  offered.  In  2002,  the  Navy  created  the  Program 
Executive  Office,  Integrated  Warfare  Systems  (PEO  IWS),  and  put  the  PEO  IWS  in 
charge  of  implementing  the  Navy‘s  OA  strategy.  This  included  the  adoption  of  standards, 
products,  and  best  practices  that  allowed  for  systems  integration  and  future  technological 
upgrades.  The  PEO  IWS  has  since  developed  and  implemented  its  own  open  architecture 
policy  called  Naval  Open  Architecture  (NOA).  The  NOA  policy  is  -a- Navy  initiative  for 
a  multi-faceted  strategy  providing  a  framework  for  developing  joint  interoperable 
systems  that  adapt  and  exploit  open-system  design  principles  and  architectures”  (Defense 
Acquisition  University,  2006).  NOA  established  a  framework  with  a  set  of  principles, 
including  the  following: 
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•  provide  more  opportunities  for  competition  and  innovation 

•  rapidly  field  affordable,  interoperable  systems 

•  minimize  total  ownership  cost 

•  optimize  total  system  perfonnance 

•  yield  systems  that  are  easily  developed  and  upgradeable 

•  achieve  component  software  reuse 

NOA  is  a  systems  design  approach  supported  by  governmental  testing  platforms 
such  as  the  Open  Architecture  Computing  Environment  (OACE).  The  OACE  is  a 
standards-based  computing  infrastructure  used  by  Surface  Command  and  Control  domain 
software  applications  that  attempts  to  implement  open  specifications  in  interfaces  and 
services.  The  OACE  is  a  compatible  set  of  standards-based  COTS  components  that 
provides  the  framework  for  which  support  applications  are  built  under  the  guidelines  of 
OA  (Department  of  Defense  [DoD],  NAVSEA,  &  PEO  IWS,  2004). 

A  few  of  the  technologies  that  guide  OACE  include  the  use  of  middleware  and 
wrappers.  Middleware  is  important  in  software  development,  particularly  in  the  context 
of  enterprise  application  integration.  Middleware  is  a  way  of  making  separate 
applications  communicate  with  one  another  without  actually  being  integrated.  It  is  the 
software  infrastructure  that  is  intended  to  support  the  deployment  of  core,  mission-critical 
applications  (Minoli,  2008).  Middleware  provides  proven  ways  to  connect  the  various 
software  components  in  an  application  so  they  can  exchange  information  using  relatively 
easy-to-use  mechanisms.  Middleware  is  completely  hidden  from  the  perspective  of  the 
application  user  (Gorton,  2006).  The  tenn  middleware  is  most  often  used  to  describe 
support  software  that  facilitates  interactions  between  major  software  components  and 
masks  the  differences  in  language,  platform  characteristics,  message  fonnats, 
communication  protocols,  data  structures,  and  other  factors  (Department  of  Defense 
[DoD],  NAVSEA,  &  PEO  IWS,  2004). 

A  wrapper  is  software  that  is  used  to  insulate  applications  from  the  applications 
programmer  interface  (API)  of  another  set  of  software  by  exporting  a  different  API.  The 
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wrapper  exposes  the  legacy  application^  functionality  or  data  to  the  SOA  as  a  service. 
The  wrapper  provides  all  the  security,  quality  of  service,  and  service  orientation 
principles  that  is  provided  by  any  other  SOA  service.  Wrappers  provide  a  way  to  reuse 
applications  already  delivering  business  value. 

In  order  for  the  Navy  to  implement  OA,  it  first  had  to  develop  an  NOA  strategy 
that  included  a  vision  statement,  principles,  goals,  and  supporting  objectives.  The  NOA 
vision  statement  is  to  -transform  our  organization  and  culture  and  align  our  resources  to 
adopt  and  institutionalize  open  architecture  principles  and  processes  throughout  the  naval 
community  in  order  to  deliver  more  warfighting  capabilities  to  counter  current  and  future 
threats”  (Program  Executive  Office,  Integrated  Warfare  Systems  [PEO  IWS],  2007). 
Figure  1  describes  the  Department  of  the  Navy  (DON)  OA  Strategy. 
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Figure  1.  DON  OA  Strategy 

(As  cited  in  Uchytil,  2006) 

In  order  to  implement  NOA,  a  Naval  Open  Architecture  Enterprise  Team  (OAET) 
was  established.  Since  the  PEO  IWS  was  assigned  overall  responsibility  for  the  NOA 
implementation,  it  was  designated  as  the  OAET  lead.  One  of  the  outcomes  of  the  OAET 
was  the  development  of  the  Open  Architecture  Assessment  Model  (OAAM).  This  model 
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provides  a  program  manager  with  a  way  to  describe  the  -openness”  of  his  current  or 
proposed  system.  In  order  to  measure  the  openness,  a  program  manager  must  use  an 
Open  Architecture  Assessment  Tool  (OAAT),  which  is  an  analytical  tool  that  assesses  the 
openness  of  a  system  based  on  business  and  technical  interrelated  questions. 

There  are  several  benefits  and  drawbacks  to  OA.  The  one  benefit  most  often 
discussed  is  the  reduction  in  life-cycle  costs.  Costs  could  be  reduced  due  to  several  of  the 
attributes  already  described  in  this  section,  such  as  modularity  and  reuse.  Because  there  is 
commonality  between  systems,  maintenance  costs  would  also  decrease.  In  addition, 
competition  from  industry  would  increase  and  thereby  drive  down  the  cost  of  upgradable 
parts.  Along  with  lower  life-cycle  costs,  other  advantages  to  OA  include  better  system 
perfonnance  due  to  easier  upgradability  with  the  latest  technologies,  as  well  as  improved 
interoperability  for  joint  warfighting. 

Although  there  are  several  advantages,  OA  has  its  disadvantages  as  well.  One  of 
the  biggest  concerns  is  security.  Although  OA  is  already  in  use  in  industries  such  as 
banking,  which  requires  a  great  deal  of  security,  there  is  no  industry  comparison  to  the 
security  required  for  a  weapons  system.  In  this  case,  careful  testing  would  be  required 
because  there  is  no  room  for  error  in  DoD  weapons  systems.  Security  and  performance 
requirements  are  typically  much  higher  in  the  DoD  than  in  industry,  so  any  COTS 
products  used  must  be  analyzed  carefully  before  implementation  to  ensure  that  they  do 
not  leave  the  network  vulnerable  to  outside  attack.  Furthennore,  added  security  measures 
typically  have  a  negative  impact  on  performance,  which  may  lead  to  the  COTS  products 
not  performing  as  well  as  advertised. 

Another  major  concern  is  cost.  Although  the  life-cycle  costs  should  be  decreased 
with  OA,  the  up-front  costs  are  very  large.  Because  it  would  not  be  feasible  to  change  the 
system  overnight,  much  of  the  cost  would  occur  during  the  transition  period.  Initially, 
there  would  be  the  cost  of  the  new  architecture.  In  addition,  there  would  be  the 
requirement  to  continue  utilizing  some  of  the  existing  legacy  systems.  This  would  mean 
the  added  cost  of  middleware  to  interface  between  the  legacy  and  OA  systems.  Also, 
during  the  transition  period,  there  would  be  maintenance  costs  incurred  for  both  systems 
simultaneously.  Eventually,  the  legacy  systems  would  be  phased  out  and  replaced  with 
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OA  systems,  but  this  might  take  a  while.  There  are  many  theories  as  to  how  long  the 
transition  might  take,  but  when  the  chief  technical  officer  of  the  DoD  Business  Mission 
Area  was  asked,  he  replied,  At  will  take  a  generation”  (Bradley,  2007).  Furthermore, 
training  would  need  to  be  implemented  in  order  to  develop  expertise  in  the  new 
architecture.  In  all,  although  life-cycle  costs  would  decrease,  costs  would  most  likely 
increase  in  the  near  term. 

C.  PRINCIPLES  OF  NAVY  OA 

To  achieve  the  vision  of  NOA,  five  principles  were  identified  by  the  Office  of  the 
Chief  of  Naval  Operations  Staff  (OPNAV),  Warfare  Requirements  and  Programs 
(N6/N7).  These  principles  are  as  follows:  encouraging  competition  and  collaboration, 
modular  design  and  design  disclosure,  interoperable  joint  warfighting  applications  and 
secure  information  exchange,  reusable  application  software,  and  life-cycle  affordability. 

1.  Encouraging  Competition  and  Collaboration 

OA  naturally  encourages  competition  and  collaboration.  Unlike  systems  that  are 
acquired  sole-source  and  restrict  the  full  and  open  competition  of  resources,  OA 
promotes  competition  among  industries,  leading  to  better  products  at  a  reduced  price.  In 
addition,  since  open  standards  are  used,  competition  in  industry  can  be  leveraged  when 
completing  system  upgrades  or  when  fielding  an  entirely  new,  but  interoperable  system. 

2.  Modular  Design  and  Design  Disclosure 

Modularity  is  the  concept  of  decomposing  a  system  into  subcomponents.  These 
subcomponents  do  not  rely  on  another  aspect  of  the  system.  In  that  way,  they  can  change 
quickly  and  allow  for  interactions  with  other  systems.  This  would  allow  for  the 
independent  upgrade  of  subcomponents,  instead  of  changing  out  an  entire  system. 
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3.  Reusable  Application  Software 

Reusable  application  software  allows  a  system  to  use  the  same  components  and 
code  that  has  been  used  in  other  platfonns.  Since  the  code  has  already  been  tested, 
certified,  and  approved,  software  reuse  would  save  both  time  and  money  compared  to 
developing  new  software  independently. 

4.  Interoperable  Joint  Warfighting  Applications  and  Secure  Information 
Exchange 

This  principle  involves  using  common  services,  common  warfighting 
applications,  and  information  assurance,  and  it  requires  these  commonalities  for  the  basic 
design  elements  of  any  new  system  (Department  of  Defense  [DoD],  NAVSEA,  &  PEO 
IWS,  2004). 


5.  Life-Cycle  Affordability 

This  principle  includes  all  life-cycle  costs  of  system  design,  development, 
delivery,  and  support.  Since  this  thesis  is  primarily  concerned  with  cost  savings,  and  it 
has  been  determined  that  initial  costs  increase  at  implementation,  life-cycle  affordability 
represents  a  key  benefit  of  this  thesis. 

Along  with  the  five  principles  listed  previously,  several  key  attributes  are  required 
when  building  an  open  architecture  framework.  An  OA  framework  should  enable  open 
systems  to  be  designed  and  to  continually  evolve  throughout  their  life  cycle.  In  order  to 
accomplish  this,  OA  provides  a  group  of  core  concepts  that  must  be  addressed.  These 
concepts  provide  the  foundation  for  an  OA  framework.  Although  not  entirely 
encompassing,  four  core  concepts  are  modularity,  reuse,  scalability,  and  portability. 
Modularity  and  reuse  have  already  been  identified. 

6.  Scalability 

Scalability  within  OA  refers  to  the  ability  to  add  new  functionalities  or  resources 
without  a  major  change  or  modification  to  the  system.  The  ability  to  add  new 
components,  update  current  ones,  or  adjust  the  scale  of  the  system  with  little  disruption  to 
the  systems  operations  is  the  basic  premise  of  the  scalability  attribute. 
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7.  Portability 

Portability  refers  to  being  able  to  move  hardware  or  software  from  one  platform 
to  another.  Proper  implementation  of  portability  into  an  OA  would  allow  for  easy 
transition  between  many  hardware  and  software  platforms  (Uchytil,  2006). 

These  core  concepts  are  especially  critical  in  today4 s  world,  where  the  rate  of 
technological  advancement  is  higher  than  it  has  ever  been. 

In  order  to  accomplish  these  principles,  the  Navy  established  three  primary  goals, 
each  of  which  has  several  subsets.  The  three  primary  goals  are  as  follows  (-Naval  OA 
Strategy,”  2008): 

1 .  Change  naval  process  and  business  practices  to  utilize  open  systems 
architectures  in  order  to  rapidly  field  affordable,  interoperable  systems. 

2.  Provide  naval  OA  systems  engineering  leadership  to  field  common, 
interoperable  capabilities  more  rapidly  at  reduced  costs. 

3.  Change  Navy  and  Marine  Corps  cultures  to  institutionalize  OA  principles. 

D.  SERVICE-ORIENTED  ARCHITECTURE  (SOA) 

In  this  section,  the  researcher  offers  several  definitions  of  SOA,  outlines  SOA 
concepts  and  principles,  and  describes  some  benefits,  as  well  as  challenges,  of  SOA. 

1.  Definitions 

The  term  service-oriented  architecture  has  no  centrally  defined  meaning.  Several 
organizations  have  provided  definitions,  but  no  concrete  definition  has  been  agreed  upon. 
Even  though  the  exact  definition  of  SOA  is  elusive  in  the  infonnation  technology 
industry,  there  are  some  basic  and  useful  concepts  that  are  generally  accepted. 

Hewlett-Packard  (HP)  defines  SOA  as:  an  architectural  approach — built  upon  the 
concept  of  software  services — for  designing,  building,  and  managing  the 
distributed  computing  infrastructure  that  an  enterprise  requires  to  execute  and 
achieve  business  strategy  and  goals.  This  approach  promotes  the  use  of  loosely 
coupled,  reusable,  standards-based,  and  well-defined  services  in  a  way  that 
enables  them  to  be  discovered  on  the  network  and  used  by  other  applications  or 
end  users.  (2005) 
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IBM  defines  SOA  as:  an  IT  architectural  style  that  supports  the  transformation  of 
your  business  into  a  set  of  linked  services,  or  repeatable  business  tasks,  that  can 
be  accessed  when  needed  over  a  network.  This  may  be  a  local  network,  it  may  be 
the  Internet,  or  it  may  be  geographically  and  technologically  diverse,  combining 
services  in  New  York,  London,  and  Hong  Kong  as  though  they  were  all  installed 
on  your  local  desktop.  These  services  can  coalesce  to  accomplish  a  specific 
business  task,  enabling  your  business  to  quickly  adapt  to  changing  conditions  and 
requirements,  (n.d.) 

Business  Transfonnation  Agency  (BTA)  defines  SOA  as:  a  way  of  describing  an 
environment  in  tenns  of  shared  mission  and  business  functions  and  the  services 
that  enable  them.  The  Government  Accountability  Office  (GAO)  describes  SOA 
as  an  =approach  for  sharing  functions  and  applications  across  an  organization  by 
designing  them  as  discrete,  reusable,  business-oriented  services.4  (GAO,  2006; 
Business  Transformation  Agency  [BTA],  2009) 

Essential  Software  Architecture  defines  SOA  as:  an  approach  to  building  software 
systems  from  independent  applications  that  communicate  only  by  accessing  the 
business-level  services  that  each  application  provides.  (Gorton,  2006) 


Although  there  are  various  definitions  of  SOA,  they  all  refer  to  services  in  one 
way  or  another.  A  definition  of  a  service  is  -an  implementation  of  a  well-defined  piece  of 
business  functionality,  with  a  published  interface  that  is  discoverable  and  can  be  used  by 
service  consumers  when  building  different  applications  and  business  processes” 
(0‘Brien,  Bass,  &  Merson,  2005,  p.  1). 

2.  Principles 

A  common  set  of  principles  most  often  associated  with  SOA  include  the 
following: 


a.  Services  are  Reusable 

Services  are  designed  to  support  potential  reuse,  regardless  of  whether 
immediate  reuse  opportunities  exist.  By  applying  standards  that  allow  reuse,  the  chances 
of  accommodating  future  requirements  with  less  development  effort  are  increased  (Erl, 
2005a). 
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b. 


Services  Share  a  Formal  Contract 


Service  contracts  provide  a  fonnal  definition  of  service  endpoint,  each 
service  operation,  and  every  input  and  output  message  supported  by  each  operation. 
Furthennore,  service  contracts  include  rules  and  characteristics  of  the  service  and  its 
operations.  In  order  for  services  to  interact,  a  formal  contract  is  needed  to  define  the 
terms  of  information  exchange.  Therefore,  service  contracts  define  almost  all  the  primary 
parts  of  an  SOA.  This  infonnation  establishes  the  agreement  made  by  a  service  provider 
and  service  requestors  (Erl,  2005a). 

c.  Services  are  Loosely  Coupled 

Loose  coupling  maintains  that  for  services  to  interact,  they  must  be  aware 
of  one  another‘s  existence.  Awareness  is  achieved  through  service  descriptions,  which 
establish  a  name  of  the  service,  a  description  of  the  data  expected  by  the  service,  and  a 
description  of  any  data  returned  by  the  service  (Erl,  2005b).  Additionally,  loose  coupling 
maintains  that  each  service  should  be  self-contained,  adding  a  level  of  abstraction  and 
service  autonomy.  Finally,  due  to  low  inter-module  dependency,  an  advantage  to  loosely 
coupled  systems  is  that  they  tend  to  have  a  shorter  development  time. 

d.  Services  Abstract  Underlying  Logic 

The  serviced  description  is  the  only  part  of  a  service  that  is  visible  to  the 
outside  world.  In  SOA,  aside  from  what  is  expressed  in  the  description  and  fonnal 
contract,  the  underlying  logic  is  invisible  and  irrelevant  to  the  service  requestors. 

e.  Services  are  Composable 

Groups  of  services  can  be  assembled  to  form  composite  services.  This 
possibility  allows  logic  to  be  represented  at  different  levels  of  granularity  and  promotes 
reusability  and  the  creation  of  abstract  layers  (0‘Brien  et  ah,  2005). 

/.  Services  are  Autonomous 

Services  have  control  over  the  logic  they  encapsulate.  The  logic  governed 
by  a  service  resides  within  an  explicit  boundary.  Within  this  boundary,  the  service  has 
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complete  autonomy,  and  it  is  not  dependent  on  any  other  service.  This  freedom  of 
dependency  eliminates  ties  that  could  inhibit  its  deployment  and  evolution  (Erl,  2005a). 

g.  Services  are  Stateless 

Services  should  not  manage  state  information,  as  that  may  impede  their 
ability  to  remain  loosely  coupled.  Services  should  be  designed  to  maximize  statelessness 
(Erl,  2005b).  A  stateless  condition  for  services  is  one  that  promotes  reusability  and 
scalability  attributes. 

h.  Services  are  Discoverable 

Services  should  allow  their  descriptions  to  be  discovered  and  understood 
by  humans  and  service  requestors  so  that  they  may  be  able  to  make  use  of  their  logic. 
Because  each  operation  provides  a  potentially  reusable  piece  of  processing  logic,  the 
service  needs  to  discover  both  the  serviced  purpose  as  well  as  the  functionality  offered 
by  its  operations  (Erl,  2005a).  Services  should  be  designed  to  be  outwardly  descriptive, 
so  they  can  be  found  and  accessed  by  availability  discovery  mechanisms.  This  service 
discovery  can  be  facilitated  by  the  use  of  a  directory  provider. 

i.  Services  are  Modular 

Although  often  covered  under  the  principle  of  loosely  coupled,  modularity 
deserves  its  own  description.  Modularity  allows  the  logic  required  to  solve  large 
problems  to  be  better  constructed,  carried  out,  and  managed  if  it  is  decomposed  into  a 
collection  of  smaller,  related  pieces  (Erl,  2005a).  Each  piece  addresses  a  specific  part  of 
the  problem,  but  when  coupled,  solves  the  larger  problem.  An  often-used  analogy  that 
distinguishes  the  traditional  architectural  approach  from  the  loosely  coupled,  modular 
design  offered  by  SOA  is  to  think  of  traditional  architecture  as  a  jigsaw  puzzle,  tightly 
coupled,  and  SOA  as  tangram  puzzles,  which  are  loosely  coupled.  Figure  2  provides  an 
example  of  tight  coupling  versus  loose  coupling. 


16 


Before  SOA 


After  SOA 
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Figure  2.  Before  &  After  SOA 

(As  cited  in  Adler  &  Ahart,  2007) 


Although  all  of  the  principles  described  in  this  section  apply  to  SOA,  autonomy, 
loose  coupling,  abstraction,  and  the  need  for  a  formal  contract  are  often  considered  the 
core  principles  that  establish  the  foundation  of  SOA  (0‘Brien  et  al.,  2005). 

3.  Attributes 

The  principles  described  in  the  previous  section  lead  to  a  set  of  quality  attributes 
in  the  context  of  SOA. 


a.  Interoperability 

Interoperability  refers  to  the  ability  of  a  collection  of  communicating 
entities  to  share  specific  information  and  to  operate  on  it  according  to  an  agreed-upon 
standard.  In  general,  interoperability  requires  some  form  of  interchange  between  two  or 
more  entities  (Brownsword  et  al.,  2004).  This  allows  common  services  to  interact 
between  new  and  legacy  systems,  regardless  of  specific  characteristics.  In  addition, 
products  from  various  vendors  are  able  to  operate  successfully  with  each  other.  SOA 
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allows  data  sharing  between  systems  that  were  unable  to  communicate  previously. 
Increased  interoperability  is  the  most  prominent  benefit  of  SOA,  especially  when  we 
consider  web  services  technology  (McGovern,  Tyagi,  Stevens,  &  Matthew,  2003). 
Finally,  interoperability  is  directly  related  to  the  concept  of  reuse.  As  more  services  are 
reused,  interoperability  increases,  providing  a  less  burdensome  IT  structure. 

b.  Reliability 

Simply  stated,  reliability  is  the  ability  of  a  system  to  keep  operating  over 
time  (Clements,  Kazman,  &  Klein,  2002).  Many  aspects  related  to  reliability  are 
important  within  SOA,  particularly  the  reliability  of  the  messages  exchanged  and  the 
reliability  of  the  services  themselves.  This  can  be  of  concern  because  different  vendors 
may  have  different  reliability  requirements  for  their  products,  and,  as  the  saying  goes,  a 
chain  is  only  as  strong  as  its  weakest  link. 

c.  Availability 

Availability  is  the  degree  to  which  a  system  is  accessible  when  it  is 
required  for  use.  SOA  provides  the  advantage  of  constant  availability  since  single 
components  are  responsible  for  compartmentalized  data.  However,  since  services  are 
loosely  coupled,  if  one  service  goes  down,  all  other  services  that  rely  on  that  given 
service  are  affected.  In  this  way,  an  entire  system  could  be  degraded.  Therefore,  when 
designing  an  SOA  around  critical  systems,  a  backup  should  be  considered  (Brummett  & 
Finney,  2008). 


cl.  Usability 

Usability  is  the  measure  of  the  quality  of  a  user‘s  experience  in  interacting 
with  the  service  or  infonnation  provided  (0‘Brien  et  al.,  2005).  A  usable  service  is 
therefore  one  that  provides  a  familiar  feel  and  requires  less  training  for  a  user  to  leam. 

e.  Security 

Security  within  SOA  is  of  vital  concern  to  the  DoD.  Generally,  security 
involves  four  main  principles:  confidentiality,  authenticity,  integrity,  and  availability.  The 
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system  must  provide  a  certain  level  of  trust  that  the  information  being  accessed  is  from  an 
authorized  user.  In  addition,  stronger  security  mechanisms  often  have  a  negative  impact 
on  perfonnance.  For  these  reasons,  security  of  SOA  is  considered  a  prime  disadvantage 
and  will  be  covered  under  challenges  of  SOA. 

/.  Performance 

Perfonnance  is  related  to  response  time  (how  long  it  takes  to  process  a 
request),  throughput  (how  many  requests  can  be  processed  per  unit  time),  and  timeliness 
(the  ability  to  process  a  request  in  an  acceptable  amount  of  time)  (0‘Brien  et  ah,  2005). 
With  SOA,  services  may  be  spread  over  a  vast  area.  This  may  affect  perfonnance  of  the 
system  with  respect  to  latency.  Furthermore,  latency  is  correlated  with  the  number  of 
times  a  service  is  invoked. 

g.  Scalability 

Scalability  is  the  ability  of  the  system  to  be  changed  in  size  or  volume  to 
meet  increased  user  demand  without  any  degradation  to  other  quality  attributes. 

h.  Extensibility 

Extensibility  refers  to  the  ease  with  which  new  services  can  be  added. 
Extensibility  becomes  vital  in  today4  s  rapidly  changing  technology  environment. 
Furthennore,  services  should  be  able  to  be  added  without  affecting  performance  of  other 
attributes  or  the  user‘s  interface,  unless  desired. 

i.  Adaptability 

Adaptability  is  the  degree  to  which  existing  services  can  be  altered  to 
better  accommodate  changing  user  requirements.  As  with  extensibility,  adaptability 
allows  the  system  to  stay  current  with  rapidly  changing  technologies,  changing 
environments,  and  changing  missions. 
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j.  Testability 

Testability  is  the  degree  to  which  a  service  can  be  tested  against  a  set  of 
criteria,  and  the  performance  of  that  service  against  those  set  criteria.  Testing  can  be 
complex  for  several  reasons,  including  the  fact  that  the  service  may  act  differently  once 
coupled  with  other  services.  Trying  to  replicate  all  the  issues  a  service  may  face  in  a  test 
environment  is  extremely  difficult.  Within  the  DoD,  testing  of  weapons  platforms  is  done 
extensively  in  expensive  testing  facilities.  As  services  move  to  connect  fonnerly  stove- 
piped  platfonns,  testability  becomes  a  critical  attribute  to  ensure  the  systems  remain 
functioning  as  they  were  meant  to  (0‘Brien  et  ah,  2005). 

k.  Modifiability 

Modifiability  is  the  ability  to  make  changes  to  a  system  quickly  and  cost- 
effectively  (Clements  et  al.,  2002).  Modifiability  tends  to  be  a  by-product  of  other  SOA 
attributes.  Because  services  are  loosely  coupled,  self-contained,  and  modular,  they  tend  to 
be  modified  rather  quickly,  easily,  and  at  a  reduced  cost. 

4.  Technology  and  Standards. 

SOA  offers  electronic  services  across  the  web,  called  web  services.  Web  services 
do  not  expose  their  implementations  to  clients,  only  their  capabilities.  The  client 
application  invokes  the  functionality  of  a  web  service  by  sending  it  messages,  receives 
return  messages,  and  uses  the  results  within  the  clients1  applications.  One  key  benefit  of 
web  services  is  that  they  are  based  on  open  standards.  This  allows  web  services  to  be 
implemented  in  any  language  and  on  any  platfonn  and  still  be  compatible  with  client 
applications.  With  this  in  mind,  a  few  technical  tenns  encountered  in  the  core  set  of  SOA 
standards  are  defined. 


a.  Extensible  Markup  Language  (XML) 

XML  is  a  language  for  marking  up  data  so  that  information  can  be 
exchanged  between  applications  and  platforms.  SOA  is  made  possible  by  the  widespread 
acceptance  of  open  standards,  and  XML  is  the  common  language  used  by  nearly  all  web 
services. 
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b.  Simple  Object  Access  Protocol  (SOAP) 

For  data  to  be  transferred  between  computers,  communication  protocols 
must  be  established.  SOAP  is  a  messaging  protocol  for  transporting  information  and 
instructions  within  a  distributed  environment  using  XML  as  a  foundation  for  the  protocol. 
SOAP  is  the  most  commonly  used  transport  protocol  standard  for  moving  messages 
between  services. 


c.  Web  Service  Description  Language  (WSDL) 

WSDL  is  an  XML  based  language  for  describing  web  services  and  for 
publishing  their  interfaces  to  the  network.  WSDL  enables  a  client  application  to 
detennine  the  location  of  the  web  service,  the  functions  it  implements,  and  how  to  access 
and  use  each  function.  The  WSDL  serves  as  a  contract  between  the  web  service  and  a 
consumer  or  potential  consumer  of  that  service.  The  WSDL  file  describes  both  the  data  to 
be  passed  and  the  method  for  passing  the  data. 

d.  Web  Service  Stack 

The  web  services  stack  shows  the  collection  of  computer  networking 
protocols  that  define,  locate,  implement,  and  make  web  services  interact  with  each  other. 
The  World  Wide  Web  Consortium^  Web  Services  Architecture  Working  Group  defined 
technical  standards  to  ensure  interoperability  for  SOAs. 

5.  Benefits 

SOA  has  several  key  advantages,  as  well  as  several  challenges.  Benefits  are 
primarily  the  result  of  the  principles  that  guide  SOA.  First,  SOA  promotes  software  reuse, 
which  reduces  design  time  and  implementation  time,  and  results  in  an  overall  cost 
reduction.  Since  the  applications  are  loosely  coupled,  testing  of  applications  can  be  done 
independently  on  the  application  itself  without  affecting  the  entire  system.  In  addition, 
service  orientation  attempts  to  solve  problems  of  the  past  by  using  the  following  concepts 
(Erl,  2008a): 

•  increased  consistency  in  how  functionality  and  data  are  represented, 

•  reduced  dependencies  between  units  of  solution  logic, 
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•  reduced  awareness  of  underlying  solution  logic  design  and  implementation 
detail, 

•  increased  opportunities  to  use  a  piece  of  solution  logic  for  multiple 
purposes, 

•  increased  opportunities  to  combine  units  of  solution  logic  into  different 
configurations, 

•  increased  behavioral  predictability, 

•  increased  availability  and  scalability,  and 

•  increased  awareness  of  available  solution  logic. 

6.  Challenges 

The  following  are  some  of  the  challenges  that  SOA  systems  face  (Erl,  2008a,  p. 


Increased  performance  requirements.  As  multiple  systems  reuse  a  single  service, 
system  performance  needs  to  increase  to  keep  up  with  demand  and  prevent 
latency  issues.  Perfonnance  measures  will  need  to  be  developed  for  each  service 
based  on  intended  usage. 

Reliability  due  to  concurrent  usage.  A  service  may  exhibit  reduced  reliability  as 
more  than  one  system  is  requiring  that  serviced  functions  at  the  same  time. 
Controls  to  mitigate  the  risk  of  reduced  reliability  must  be  introduced  for  critical 
systems. 

Single  point  failure.  As  an  increasing  number  of  systems  rely  on  one  service  for  a 
particular  function  or  process,  failure  of  the  service  will  impact  every  system 
relying  upon  that  service.  Governance  may  aid  in  mitigating  this  risk.  Backup 
systems  are  not  ideal,  but  should  be  considered  for  high-risk  processes. 

Increased  demand  on  hosting  environments.  As  demand  on  hosting  environments 
increases,  runtimes  may  become  excessive  and  lead  to  excessive  latency  issues. 
Hosting  environments  will  need  to  be  scalable  to  mitigate  increased  demand. 
Concurrent  requests  from  multiple  applications  must  be  addressed  to  reduce 
latency  issues  as  a  service  processes  these  requests. 

Service  contract  versioning  issues  and  redundant  service  contracts.  Service 
contracts  address  how  services  will  interface  with  various  applications  and 
describe  their  desired  functionality.  Versioning  must  be  standardized  to  avoid 
confusion  and  redundant  operations  that  may  lead  to  increased  runtime.  Proper 
governance  will  reduce  the  likelihood  of  versioning  issues  and  redundant  service 
contracts. 

Security  across  the  architecture.  While  the  loose  coupling  of  the  network 
connections  between  service  requester  and  service  provider  gives  the  global 


architecture  resilience  in  recovery  from  intrusion,  it  also  means  that  the  system, 
much  the  same  as  the  Internet,  is  virtually  unbounded,  and  the  number  of  users 
accessing  services  is  unknown.  Unnecessary  requests  for  service  or  unauthorized 
service  requests  could  go  undetected,  using  up  valuable  bandwidth  and  possibly 
compromising  the  confidentiality  of  information  without  the  networks4  owners 
discovering  the  loss  until  it  is  too  late  to  recover.  (Teply,  2009,  p.  38) 


23 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


24 


III.  RELATIONSHIP  BETWEEN  OA  AND  SOA  AND  HOW  SOA 
ACCOMPLISHES  NOA  STRATEGY 


Many  of  the  same  principles  of  NOA  defined  in  Chapter  II  are  replicated  in  the 
principles  of  SOA  used  in  industry.  Table  2  compares  some  of  the  open  systems 
characteristics  from  Table  1  with  OA  concepts  used  in  the  Navy  and  SOA  concepts  used 
in  industry. 


Table  2.  Comparison  of  Open  Systems  to  OA  and  SOA 


Open  System 
Characteristics 

Open  Architecture 
Characteristics 

Service-Oriented 

Architecture 

Characteristics 

Heavy  emphasis  on 
modularity 

Modular  design  and  design 
disclosure 

Services  are  modular 

Lower  total  ownership 
cost  and  systems  with 
longer  life  expectancy 

Life-cycle  affordability 

Reliability  and 
modifiability  attributes 
decrease  cost  over  the 
lifetime  of  the  system 

Easier,  quicker,  and  less 
expensive  expansion 
and  upgrading 

Easily  upgradable  systems 

Adaptability,  extensibility, 
and  modifiability  all 
contribute  to  ease  of 
upgrading  a  system 

High  degree  of 
portability,  connectivity, 
interoperability,  and 
scalability 

Core  concepts  of  scalability  and 
portability,  and  stated  goal  of 
interoperability 

Quality  attributes  of 
scalability  and 
interoperability 

Faster  and  less  costly 
technology  transfer 

Goal  to  optimize  system 
perfonnance 

Quality  attribute  of 
perfonnance 

Reusable  application  software 

Reusable  services 

Interoperable  joint  warfighting 
applications  and  secure 
information  exchange  (common 
services  and  information 
assurance) 

Quality  attributes  of 
usability  (common 
services)  and  security 

Note :  This  table  was  adapted  from  a  similar  table  in  Azani  (2001). 
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As  many  of  the  principles  are  similar,  it  is  possible  to  treat  them  as  like  concepts 
for  the  purpose  of  this  thesis.  Therefore,  successes  and  failures  resulting  from 
implementing  SOA  in  industry  should  be  similar  to  the  expected  outcomes  of 
implementing  OA  in  the  Navy.  Among  the  outcomes  that  can  be  compared  is  the 
potential  for  cost  savings. 

As  shown  in  Table  2,  SOA  and  OA  are  much  alike.  In  fact,  the  principles  laid  out 
by  the  Defense  Acquisition  System,  which  guides  the  procurement  of  systems  for  the 
DoD,  also  resemble  several  of  the  same  principles  used  in  SOA.  Furthermore,  there  is 
already  a  practice  in  place  for  implementing  an  open  architecture  in  the  DoD,  whose 
goals  also  closely  follow  the  goals  of  SOA.  This  is  the  Modular  Open  Systems  Approach 
(MOSA).  Both  concepts  are  presented  next. 

A.  DEFENSE  ACQUISITION  SYSTEM 

The  Defense  Acquisition  System  is  a  complex,  multi-faceted  system  used  by  the 
DoD  for  the  acquisition  of  its  national  security  systems.  As  laid  out  in  DoDD  5000.01 
(USD[AT&L],  2003),  five  fundamental  principles  govern  the  Defense  Acquisition 
System.  The  five  principles  are  flexibility,  responsiveness,  innovation,  discipline,  and 
streamlined  and  effective  management.  Each  policy  directive  can  be  supported  by  SOA 
and  OA. 

1.  Flexibility 

Flexibility  is  achieved  by  both  SOA  and  OA  through  increased  agility  and  the 
potential  for  reuse.  The  more  open  the  system  becomes,  the  more  quickly  the  system  can 
adapt  to  changing  needs  or  requirements,  thereby  increasing  overall  flexibility. 

2.  Responsiveness 

SOA  and  OA  provide  the  necessary  responsiveness  by  deploying  systems  to  the 
warfighter  in  the  shortest  time  practicable.  Although  a  mature  SOA  or  OA  system  is 
required  for  maximum  responsiveness,  the  principle  of  responsiveness  will  be  achieved 
through  attributes  such  as  modifiability  and  adaptability. 
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3. 


Innovation 


Program  managers  should  adopt  innovative  practices  to  include  best  commercial 
practices  that  reduce  cycle  time  and  cost.  This  can  be  accomplished  by  OA,  since  SOA 
practices  are  proven  in  commercial  industry.  OA  is  intended  to  reduce  costs  and 
development  times.  It  also  will  reduce  future  costs  through  reuse  and  interoperability. 
Furthennore,  cycle  time  will  be  reduced  due  to  the  reduction  in  redundant  DoD  systems. 

4.  Discipline 

The  same  level  of  discipline  that  applies  to  all  acquisitions  programs  is  required 
with  OA.  However,  since  these  technologies  are  relatively  new  to  the  DoD,  standard 
baseline  parameters  and  exit  criteria  will  need  to  be  developed  with  data  from  programs 
using  this  technology. 

5.  Streamlined  and  Effective  Management 

Streamlined  and  effective  management  refers  to  the  management  of  an 
acquisitions  program,  ensuring  credibility  in  cost,  schedule,  and  perfonnance  reporting. 
SOA  and  OA  can  contribute  to  this  because  proven  technologies  have  reduced  risk, 
thereby  enhancing  the  management  of  the  overall  program. 

B.  MODULAR  OPEN  SYSTEMS  APPROACH  (MOSA) 

MOSA  is  a  way  of  implementing  open  architecture  in  the  DoD.  It  is  a  strategy  for 
developing  a  new  system  or  modernizing  an  existing  one.  It  uses  widely  supported 
commercial  interface  standards  when  developing  systems.  According  to  the  Open 
Systems  Joint  Task  Force  (OSJTF,  2004),  MOSA  attempts  to  achieve  the  following: 

•  reduced  acquisition  cycle-time  and  overall  life-cycle  cost, 

•  the  ability  to  insert  cutting-edge  technology  as  it  evolves, 

•  commonality  and  reuse  of  components  among  systems,  and 

•  an  increased  ability  to  leverage  commercial  investment. 
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In  order  to  achieve  these  benefits,  MOSA  adheres  to  five  major  principles: 
establishment  of  a  MOSA-enabling  environment,  employment  of  modular  design, 
designation  of  key  interfaces,  use  of  open  standards  for  key  interfaces,  and  certification 
of  conformance  (OSJTF,  2004).  Figure  3  identifies  the  principles  alongside  associated 
benefits  provided: 


Modular  Open  Systems  Approach 

(MOSA) 


Vision  Principles 


Benefits 


MOSA  is  an 
integral  part  of  all  1 
acquisition  strategies  to 
achieve  affordable, 
evolutionary, 
and  joint  combat 
capability  * 


Establish  Enabling  Environment 


Employ  Modular  Design 


Designate  Key  Interfaces 


Select  Open  Standards 


Certify  Conformance 


•/  Ease  of  Change 

•Z  Reduced  Total  Ownership  Cost 

Z  Reduced  Cycle-Time 

Z  Enabling  Joint  Integrated 

Architectures  and  Interoperability 

Z  Risk  Mitigation 


k 

Business  Technical 

Indicators 

Figure  3,  MOSA  Principles 

(From  OSJTF,  2004) 


The  goals  of  MOSA,  along  with  the  principles  that  guide  MOSA,  closely  relate  to 
those  strategies  that  guide  SOA.  In  addition,  some  of  the  underlying  technical  concepts 
relate  MOSA  principles  to  OACE  and  SOA  as  seen  in  Table  3. 
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Table  3.  Comparison  of  MOSA  Principles  to  OACE  and  SOA 


MOSA  Principles 

OACE 

SOA 

1 .  Establish  an  enabling 
environment  -  Establish 
supportive  requirements, 
business  practices, 
technology  development, 
acquisition,  test  and 
evaluation,  and  product 
support  strategies. 

Guidance  concerning 
standards  have  already 
been  published. 

Already  adheres  to  an 
enabling  environment 
because  many  major 
companies  are  supporting 
SOA. 

2.  Employ  modular  design  - 
Partitioned  into  scalable, 
reusable  modules.  Designed 
for  ease  of  change. 

Functional  partitioning 
should  support  insertion  of 
new  functionality. 

SOA  services  are  modular. 

3.  Designate  key  interfaces 
-  Identify  interfaces  that  are 
highly  reliable, 
technologically  stable,  and 
that  pass  vital 

interoperability  infonnation. 

Use  structured 
programming  within 
components  and 
middleware  technologies 
for  interconnections  and 
integration  among 
components. 

Use  of  wrappers  to  connect 
key  interfaces  that  must 
interoperate. 

4.  Use  open  standards  - 
Standards  must  permit 
interchangeability, 
interconnections,  and 
compatibility.  Standards 
must  be  well  defined, 
mature,  widely  used,  readily 
available,  and  allow  for 
future  technology  insertion. 

OACE  encourages 
standards-based 
technologies.  Recognizes 
XML  and  SOAP  as 
standards.  Programming 
language  should  support 
open  standards. 

Uses  open  standards  such  as 
XML,  SOAP,  and  WSDL  to 
ensure  interoperability 
among  services. 

5.  Certify  Conformance  - 
Modules  must  confonn  to 
open  interfaces  to  allow 
plug-and-play  and 
reconfiguration  of  mission 
capability  in  response  to 
new  threats  and 
technologies. 

Existing  systems  may  see 
little  if  any  change  at  the 
periphery,  but  changes  are 
made  at  the  interface. 

Web  services  are  based  on 
open  standards  and  only 
expose  their  capabilities  to 
clients,  not  their 
implementations. 

Note.  This  table  was  constructed  using  information  from  the  following  sources:  OSJTF  (2004),  and 
Department  of  Defense  [DoD],  NAVSEA,  &  PEO  1WS  (2004). 
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In  this  way,  MOSA  is  a  tool  that  guides  the  DoD  in  the  use  of  OA  in  much  the 
same  manner  as  the  principles  that  guide  the  use  of  SOA  in  industry,  further  amplifying 
the  fact  that  they  are  similar  and  can  be  treated  as  such  for  the  purpose  of  this  thesis. 
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IV.  ROI  CALCULATION  FOR  SOA 


Since  the  IT  boom  of  the  1990s,  billions  of  dollars  have  been  invested  into  IT, 
with  the  goal  of  realizing  significant  returns.  However,  returns  have  not  materialized  as 
expected,  leading  to  Nobel  Prize-winning  economist  Robert  Solow‘s  ^productivity 
paradox,”  which  explains  that  even  though  IT  is  embedded  in  more  business  processes, 
returns  are  not  showing  up  in  productivity  statistics  (Atkinson  &  Court,  2010).  Of  the 
possible  reasons  leading  to  the  productivity  paradox,  one  is  the  fact  that  people  cannot 
properly  measure  the  returns  produced  by  technology.  One  method  frequently  applied  to 
IT  systems  is  the  ROI  measurement.  ROI  is  calculated  as  the  revenue  or  benefits  of  an 
investment  minus  the  investment  cost,  divided  by  the  cost  of  the  investment.  This  figure 
is  expressed  as  a  percentage  and  is  interpreted  as  a  productivity  measure  (Nelson,  2010). 
ROI  is  an  important  measure  to  businesses,  as  evidenced  by  the  fact  that  80%  of 
companies  surveyed  by  ComputerWorld  and  Ernst  and  Young  said  the  financial 
justification  of  IT  projects  is  important.  However,  of  the  companies  surveyed,  only  40% 
perfonn  a  financial  business  case  analysis  on  a  regular  basis.  Additionally,  65%  of 
companies  indicated  they  do  not  have  the  knowledge  or  tools  needed  to  calculate  ROI, 
and  75%  said  they  have  no  fonnal  process  for  measuring  ROI  for  IT  projects.  Finally, 
68%  said  they  do  not  perform  a  follow-up  ROI  calculation  six  months  after  implementing 
the  project  (Tian,  Cao,  Ding,  Zhang,  &  Lee,  2007). 

The  ROI  for  SOA  is  considered  by  many  to  be  difficult,  if  not  impossible,  to 
calculate.  This  is  because  attributes  such  as  efficiency  are  difficult  to  quantify.  However, 
calculating  the  ROI  is  important  because  most  businesses  look  for  a  tangible  ROI  when 
they  evaluate  or  approve  new  or  continuing  investments.  A  British  study  found  that  89% 
of  companies  use  -intuition”  or  -guesswork”  to  calculate  the  ROI  of  their  IT  investments 
(DiMare,  2009,  p.  5).  According  to  ZapThink  Research,  -only  by  understanding  the  full 
range  of  SOA  value  propositions  can  companies  begin  to  get  a  handle  on  calculating  the 
ROI  of  SOA”  (Schmelzer,  2005,  para.  2).  Furthermore,  Gartner  analyst  Randy  Heffner 
has  said,  -any  attempt  to  assign  a  specific  ROI  to  SOA  should  be  viewed  with  heavy 
skepticism”  (McKendrick,  2007,  para.  3).  McKendrick  further  argued  that  SOA  is  a  set  of 
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best  practices  that  are  relatively  intangible  (McKendrick,  2007).  Some  argue  that  not  only 
should  monetary  values  define  ROI,  but  that  return  on  closing  capability  gaps  that  are 
targeted  by  SOA  implementation  and  nonmonetary  valuations  such  as  customer 
satisfaction  and  avoidance  of  loss  of  life  should  define  ROI  (Buck,  Das,  &  Hanf,  2008). 
Figure  4  displays  some  nonmonetary  considerations  for  analyzing  ROI. 
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Figure  4.  ROI  Analysis  Considerations  for  SOA 

(From  Buck  et  al.,  2008) 


There  is  an  old  adage  that  you  cannot  manage  what  you  cannot  measure.  Since 
SOA  is  made  up  of  a  variety  of  service  components  that  only  show  their  true  value  when 
working  together,  measuring  the  ROI  for  SOA  can  become  quite  convoluted.  ROI  is 
easier  to  calculate  when  using  single-purpose  applications.  Each  application  can  be 
measured  and  translated  to  an  understandable  ROI.  According  to  Erl  (2008),  -this  type  of 
reasoning  is  what  has  led  to  the  popularity  of  siloed  application  environments”  (p.  257). 

Service  reuse  adds  to  the  complexity  associated  with  calculating  the  ROI  of  SOA 
because  the  benefits  may  not  be  realized  initially.  As  a  service  is  reused,  the  ROI  will 
continue  to  increase,  as  illustrated  in  Figure  5. 
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Figure  5.  Example  of  ROI  for  SOA  Projects 

(From  Erl,  2008,  p.  62) 

Although  there  is  a  difference  of  opinion  among  experts  as  to  how  the  ROI  can  be 
calculated  within  an  SOA  implementation,  one  recommendation  is  to  divide  SOA  ROI 
calculations  into  three  quantifiable  benefits:  -f-1]  Tactical  ROI  as  a  result  of  standards- 
based  service  oriented  integration,  [2]  Operational  ROI  based  on  service  and  process 
reuse,  and  [3]  Strategic  ROI  due  to  business  and  technology  agility”  (Gabhart,  2007,  p. 
2). 

Tactical  ROI  focuses  on  reducing  redundancy  and  other  initial  cost  reductions  to 
provide  justification  for  initiating  an  SOA.  The  following  four  steps  describe  the  method 
for  calculating  tactical  ROI  (Gabhart,  2007,  p.  2): 

1 .  Compute  the  savings  realized  due  to  reduced  middleware  licensing 
costs. 

2.  Compute  the  savings  afforded  due  to  reduced  development  time. 

3.  Project  savings  due  to  reduced  maintenance  costs. 


X 

delivery  <o« 
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4.  Add  the  results  of  steps  1-3  together  and  fold  that  into  whatever 
ROI  fonnula  your  organization  uses  (i.e.,  net  gain  divided  by 
investment). 

Operational  ROI  provides  feedback  by  analyzing  the  reuse  of  services,  which 
extends  implementation  beyond  the  initial  time  frame.  Two  methods  for  calculating 
operational  ROI  for  SOA  are  the  iterative  reuse  model  and  the  calculated  reuse  model. 
When  using  the  iterative  reuse  model,  the  investment  return  is  measured  based  on  the 
number  of  times  a  service  or  process  is  reused  rather  than  an  arbitrary  time  frame” 
(Gabhart,  2007,  p.  3).  Writing  a  program  for  reuse  is  not  free.  The  relative  cost  of  writing 
a  program  for  reuse  is  approximately  1.5  times  or  50%  more  than  writing  software  for 
one-time  use  (Poulin,  1997).  Although  reusable  components  initially  cost  more  than 
nonreusable  components,  they  provide  a  cost  savings  each  time  the  service  is  reused.  The 
calculated  reuse  model  requires  that  an  organization  compare  current  development  costs 
with  the  costs  required  to  develop  reusable  components.  According  to  Gabhart  (2007), 
the  calculated  reuse  model  is  a  -mathematical  model  [that]  computes  SOA  value  based 
upon  a  few  key  variables  such  as  number  of  services  available  for  reuse,  degree  of  reuse, 
and  service  complexity”  (p.  3). 

Strategic  ROI  should  be  calculated  to  provide  a  complete  analysis  of  the  long¬ 
term  benefits  gained  by  implementing  an  SOA.  Strategic  ROI  is  described  by  Gabhart 
(2007)  in  the  following  way: 

Strategic  ROI  is  manifested  though  cost  controls,  risk  mitigation,  and  new 
revenue  generation  as  a  result  of  agility.  . . .  Strategic  ROI  is  the  ultimate 
expression  of  what  SOA  is  all  about.  It‘s  about  making  a  strategic 
investment  in  an  agile  enterprise  infrastructure  and  at  the  same  time 
aligning  the  business  and  technology  sides  of  the  organization  to  work 
toward  common,  shared  objectives,  (p.  4) 

Calculating  strategic  ROI  is  considered  more  an  art  than  a  science.  The  following 
are  some  ideas  from  Gabhart  (2007)  for  calculating  strategic  ROI: 

•  System  development  and  maintenance  costs  saved  due  to  the  ability  to  modify 
information  systems  with  little  to  no  coding  required  (simply  modify  or  rearrange 
the  orchestration  of  several  services). 
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•  Estimated  legal  costs  and  fines  avoided  due  to  faster  and  more  reliable 
responsiveness  to  regulatory  changes. 

•  Revenue  generated  via  the  rapid  creation  of  new  services  as  well  as  the 
manipulation  and  reconfiguration  of  existing  ones. 

•  Revenue  generated  due  to  ability  to  expose  internal  capabilities  as  consumable 
services  by  business  partners  and  clients  (this  potentially  generates  completely 
new  streams  of  income,  (p.  4) 

In  addition  to  Gabharfis  method,  other  methods  have  been  introduced,  such  as 
resource-consumption-based  pricing,  in  which  the  consumption  of  services  is  metered 
(Denne,  2007).  Although  experts  cannot  decide  on  one  method  of  calculating  ROI  for 
SOA,  the  previously  mentioned  methods  are  the  current  theories  on  how  to  proceed  with 
calculating  ROI  for  SOA. 

Commercial  industry  methods  for  calculating  ROI  do  not  readily  translate  to  the 
DoD  because  of  the  absence  of  profit  in  government.  Since  the  motive  is  not  profit, 
monetary  values  such  as  cost  savings,  cost  reduction,  and  cost  avoidance  are  typically 
measured  (Phillips,  2002).  However,  some  experts  argue  that  non-quantifiable  attributes 
must  be  analyzed  as  well.  These  attributes  provide  the  overall  value  associated  with 
implementing  SOA  and  must  be  taken  into  account. 

Nelson  (2010)  identifies  a  few  key  concepts  agreed  on  by  professionals  that 
contribute  to  the  difficulty  of  measuring  the  ROI  in  IT : 

•  the  difficulty  of  defining  the  actual  impact  (benefits)  of  IT  in  tenns  of  value 
because  technology  enhances  an  existing  process  or  is  embedded  within  many 
processes  that  are  stand-alone,  and 

•  the  difficulty  of  assigning  monetary  value  to  intangible  and  tangible  benefits 
(i.e.,  customer  satisfaction,  customer  retention,  or  time  savings,  (p.  17) 

There  are  several  approaches  for  addressing  these  difficulties,  with  one  such 
approach  being  the  cost-based  method.  The  cost-based  approach  was  adopted  to  try  to 
overcome  the  lack  of  a  defined  revenue  and  the  difficulties  of  assigning  monetary  value 
to  the  impact  that  is  provided  by  an  IT  investment.  This  method  is  used  when  a  profit 
margin  cannot  be  calculated  because  of  the  lack  of  a  revenue  stream.  Instead,  estimates  of 
cost  savings  are  used  as  a  surrogate  for  revenue  to  calculate  benefits.  Cost  savings  can  be 
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defined  as  the  resulting  reduction  in  expenditures  from  the  implementation  of  IT  (Nelson, 
2010).  Methods  for  calculating  cost  savings  include  the  following: 

•  Presuming  that  the  cost  to  replace  or  outsource  IT  is,  without  proof,  proportionate 
to  the  value  it  adds  to  process  performance  (Pavlou,  Housel,  Rodgers,  &  Jansen, 
2005,  p.  207). 

•  Utilizing  the  cost  reductions  that  can  be  achieved  through  staff  reductions, 
consolidation  of  facilities,  elimination  of  software  licenses,  or  other  results  that 
decrease  current  expenditures  as  cost  savings  (Brandon,  2010). 

•  Converting  output  data  to  monetary  value  by  determining  the  amount  of  impact 
the  technology  had  for  each  unit  of  cost  reduction  (Phillips  &  Phillips,  2002,  p. 
524). 

•  Calculating  the  cost  of  quality  and  directly  converting  quality  improvements  to 
cost  savings  (Phillips  &  Phillips,  2002,  p.  524). 

•  When  employee  time  is  saved,  the  participants  wages  and  benefits  are  used  for 
the  value  of  time  and  are  converted  to  cost  savings  (Phillips  &  Phillips,  2002,  p. 
524). 

All  these  cost  savings  or  cost  avoidances  serve  as  a  replacement  for  revenue  in  the  ROI 
equation  and  are  used  to  represent  the  net  benefits  or  numerator  of  the  ROI  equation.  The 
denominator  of  the  ROI  equation,  the  investment  cost,  is  calculated  by  summing  all  the 
related  costs  of  the  IT.  Sometimes,  cost  savings  is  the  only  measure  used  to  calculate 
ROI.  This  assumes  the  net  benefits  did  not  change  as  a  result  of  the  cost  reduction.  When 
the  net  benefits  or  numerator  are  held  constant,  while  reducing  costs  or  the  denominator, 
the  result  equates  to  a  positive  ROI.  Essentially,  every  time  cost  is  reduced,  ROI  is 
increased.  Using  this  logic,  the  goal  would  be  to  decrease  costs  to  zero,  thereby  resulting 
in  an  infinite  ROI  because  a  zero  would  be  in  the  denominator.  This  is  obviously 
unrealistic  because  a  company  cannot  exist  without  producing  some  type  of  cost. 
Therefore,  a  major  limitation  of  cost-based  ROI  approaches  is  that  they  rely  on  cost  to 
detennine  value.  This  creates  a  major  problem  when  estimating  ROI  because  cost  and 
revenue  need  to  be  derived  independently  in  order  to  derive  a  true  numerator.  Cost-based 
approaches  lack  a  surrogate  for  revenue  (Pavlou  et  ah,  2005). 

One  way  to  curtail  the  problems  associated  with  the  cost-based  approaches  is  the 
use  of  the  knowledge-value-added  (KVA)  methodology.  KVA  provides  surrogate 

revenue  streams  at  the  subprocess  level  that  are  uniquely  derived  from  common  units  of 
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output.  This  is  accomplished  by  providing  an  objective  method  to  estimate  value  in  terms 
of  common  units  of  output,  allowing  allocation  of  surrogate  revenue  streams  in  the 
nonprofit  sector  by  assuming  a  direct  relationship  between  knowledge  and  the  value 
stemming  from  it  and  describing  all  process  outputs  in  common  units  (Housel,  Kanevsky, 
Rodgers,  &  Little,  2009). 

According  to  Housel  and  Mun  (2010), 

KVA  measures  the  value  provided  by  human  capital  assets  by  analyzing 
an  organization,  process  or  function  at  the  process  level.  It  provides 
insights  into  each  dollar  of  IT  investment  by  monetizing  the  outputs  of  all 
assets,  including  intangible  assets  [e.g.,  assets  produced  by  IT  and 
humans].  By  capturing  the  value  of  knowledge  embedded  in  an 
organization^  core  processes  [i.e.,  employees  and  IT],  KVA  identifies  the 
actual  cost  and  revenue  of  a  process,  product,  or  service.  Because  KVA 
identifies  every  process  required  to  produce  an  aggregated  output  in  terms 
of  the  historical  process  and  cost-per-common-unit  of  output  of  those 
processes,  unit  costs  and  unit  process  can  be  easily  calculated  (p.  7).  Once 
cost  and  revenue  streams  have  been  assigned  to  sub-organizational 
outputs,  normal  accounting  and  financial  performance  and  profitability 
metrics  can  be  applied,  (p.  7) 

Although  other  methods  of  measuring  value  such  as  KVA  exist,  currently  many 
companies  use  cost-based  ROI  analysis  to  choose  a  particular  investment  option, 
considering  resource  constraints,  and  to  measure  the  ongoing  perfonnance  of  the 
investment.  However,  using  only  cost  savings  typically  does  not  tell  the  whole  story,  and 
decision  makers  must  beware  that  analysis  results  can  be  readily  manipulated  (Buck  et 
al„  2008). 
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V.  DATA  AND  ANALYSIS 


A.  METHODOLOGY 

The  researcher  gathered  data  from  a  wide  range  of  published  reports  and  surveys. 
All  reports  were  retrieved  free  of  charge  from  various  sites  on  the  Internet,  primarily 
from  company  white  papers  or  case  study  reports  that  were  accessible  after  subscribing  to 
e-mail  lists.  The  reports  consisted  primarily  of  industry-sponsored  case  studies,  and  they 
analyzed  a  particular  business  that  was  implementing  a  specific  SOA  solution  to  meet  its 
unique  objectives.  The  reports  then  assessed  the  success  or  failure  resulting  from 
implementing  an  SOA  based  on  ROI.  An  example  of  such  a  case  study  is  a  report 
sponsored  by  Hewlett  Packard  on  the  ROI  realized  by  a  company  after  incorporating  one 
of  HP‘s  SOA  services.  In  all,  34  case  studies  from  a  variety  of  business  domains  were 
reviewed  in  detail.  The  method  used  to  report  ROI  was  not  uniform  in  all  of  the  reports. 
Some  reports  broke  the  cost  savings  into  costs  avoided  or  into  productivity 
improvements,  of  which  only  a  percentage  was  provided  or  could  be  calculated,  and 
others  simply  stated  a  dollar  amount  of  cost  savings  without  including  supporting  figures. 
When  feasible,  the  researcher  broke  costs  out  into  the  three  quantifiable  areas  recognized 
in  DoD  financial  management:  (1)  cost  savings  or  actual  reduction  of  cost  in  a  current 
area;  (2)  cost  avoidance,  a  reduction  or  elimination  in  a  future  requirement;  and  (3) 
productivity  improvement,  a  reduction  in  future  personnel  time  and  effort  (American 
Society  of  Military  Comptrollers,  2009).  From  the  34  case  studies  analyzed,  18  provided 
an  overall  ROI,  and  from  those,  10  were  broken  down  into  the  various  cost  components. 
All  reports  were  used  to  draw  conclusions  about  benefits  considered  important  to  industry 
and  to  its  best  practices.  The  overall  ROI  from  industry1  s  implementation  of  SOA  was 
found  to  be  305%,  as  shown  in  Table  4,  while  the  ROI  from  cost  savings  and  cost 
avoidance  was  calculated  to  be  72%,  as  shown  in  Table  5. 
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B.  ANALYSIS 

Table  4  displays  information  taken  directly  from  the  case  studies  for  the  18 
selected  companies  with  a  reported  overall  ROI.  Any  column  left  blank  indicates  that  the 
information  was  not  presented  in  the  report.  Many  companies  declined  to  include  their 
actual  company  name  in  the  report  and  are  instead  referred  to  by  their  type  of  business. 
Because  the  case  studies  were  conducted  by  different  companies,  their  methods  for 
calculating  ROI  varied  as  well.  As  shown  in  Table  4,  ROI  was  calculated  over  a  three-  to 
six-year  period.  All  companies  calculated  a  net  present  value  (NPV)  with  a  discount  rate 
of  12%.  Furthennore,  a  payback  period  was  calculated  for  most  case  studies.  The  authors 
of  the  reports  said  ROI  was  calculated  under  a  process  of  measuring  the  benefits, 
calculating  the  total  investment,  and  then  projecting  the  investment  and  benefit  over  the 
time  period  designated.  The  reports  did  not  provide  details  on  exactly  how  benefits  were 
measured. 
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Table  4.  Baseline  Data — ROI  Reported  by  18  Selected  Companies  According  to  Case 

Study  Reports 


Company 

ROI 

Benefit 

(discounted) 

Investment 

(discounted) 

NPV 

Discount  % 

Discount 

Period 

(Years) 

Payback 

(months) 

Blue  Cross  Blue  Shield 

of  KC 

332% 

14,330,000 

3,320,000 

11,010,000 

12% 

6 

20 

Mobile  Telecom 

625% 

10,120,000 

1,400,000 

8,720,000 

12% 

3 

5.6 

Real  Time  Services 

215% 

180,000 

57,000 

120,779 

5 

0 

Global  Provider  for 

Info  Mgmt  Sys 

470% 

8,080,525 

1,417,846 

6,662,679 

12% 

3 

2.5 

Services  and  Fac  Mgmt 

Co 

360% 

2,744,982 

596,674 

2,148,309 

12% 

3 

4.6 

European  based 
telecom 

212% 

5,472,842 

1,753,242 

3,719,600 

12% 

3 

9 

International  Finance 

Firm 

252% 

$6,627,447 

$1,882,568 

$4,744,879 

12% 

3 

6.7 

Flealthcare  Provider 

356% 

$13,475,631 

$2,952,633 

$10,522,889 

12% 

6 

6.7 

Global  Media 
Consulting  Firm 

244% 

$1,541,718 

$447,938 

$1,093,780 

12% 

3 

8.2 

Healthcare  Services 

Provider 

346% 

$15,800,000 

$3,500,000 

$12,300,000 

12% 

3 

4.8 

Global  Financial 

Services  Firm 

472% 

$37,140,000 

$6,490,000 

$30,650,000 

12% 

3 

3.9 

Carphone 

42% 

$1,254,000 

$812,000 

3 

30.6 

Johnson  Controls 

81% 

$370,000 

$143,547 

3 

12 

Bank  of  India 

234% 

$23,000,000 

5 

24 

More  Direct 

428% 

$445,395 

$47,270 

$332,251 

5 

5 

International 

Insurance  Provider 

256% 

$1,428,180 

$401,607 

$1,026,573 

12% 

3 

8 

Global  Consumer 

Products  Co 

265% 

$1,118,547 

$306,370 

$812,176 

3 

5.8 

Quicken  Loans 

298% 

$183,000 

Average 

305% 

9.4 

Note :  This  table  was  constructed  using  information  from  the  following  case  studies:  Case  Study  Forum 
(2009a,  b),  IDC  Business  Value  Spotlight  (2009a,  b,  c,  d,  e,  2010a,  b,  c),  IDC  ExpertROI5  Spotlight 
(2010a,  b,  c,  d),  Shopping  for  SOA  (n.d),  Nucleus  Research  (2007,  2008),  and  Thoughtfare  Worldwide 
(2010). 

Table  5  displays  the  calculated  ROI  for  the  10  companies  that  broke  out  the 
benefits  into  categories.  The  researcher  further  broke  down  this  data  into  either  annual 
cost  savings  achieved  by  SOA  implementation,  annual  cost  avoidance,  or  annual 
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productivity  improvement.  For  the  purposes  of  this  thesis,  cost  savings  was  defined 
simply  as  the  difference  between  the  costs  historically  paid  and  the  costs  after 
implementing  an  SOA  component.  These  costs  were  tangible  benefits  that  could  be 
recorded  and  programmed  into  a  budget.  Cost  avoidance,  on  the  other  hand,  were  those 
costs  that  were  planned,  but  because  of  the  SOA  implementation,  did  not  need  to  be 
executed.  A  few  examples  of  cost  avoidance  were  not  hiring  additional  workers,  not 
outsourcing,  or  not  making  a  planned  purchase.  In  addition,  if  the  case  study  stated  the 
company  saved  some  number  of  full-time  equivalent  (FTEs)  workers,  this  was  considered 
a  cost  avoidance  because  they  no  longer  needed  to  hire  those  workers.  Although  still 
considered  a  cost  benefit,  cost  avoidance  savings  were  not  considered  true  cost  savings 
because  it  was  unclear  whether  these  future  costs  would  ever  have  been  realized.  All 
remaining  quantifiable  benefits  fell  into  the  productivity  improvement  category. 
Productivity  improvement  was  considered  the  ability  to  accomplish  more  tasks  in  the 
same  amount  of  time  by  the  some  number  of  workers.  Two  primary  examples  were  staff 
efficiency  and  improved  system  availability.  Staff  efficiency  was  calculated  as  work 
hours  saved.  If  the  position  was  eliminated  due  to  the  efficiency,  it  was  considered  a  cost 
savings;  however,  the  majority  of  the  time  the  worker  was  simply  available  to  work  on 
other  projects  and,  therefore,  was  considered  a  productivity  improvement.  System 
availability  or  reduced  downtime  was  also  calculated  on  an  hourly  basis.  The  reduced 
downtime  allowed  workers  to  continue  their  jobs  rather  than  stand  idle  while  the  system 
was  unavailable. 

To  calculate  the  ROI  from  cost  savings/cost  avoidance,  the  average  annual  cost 
savings  and  average  annual  cost  avoidance  columns  were  summed.  They  were  considered 
the  benefit.  Then,  the  ROI  was  calculated  using  Equation  1 . 

ROI  =  (Benefit  -  Cost  of  Investment)/Cost  of  Investment  Equation  (1) 

The  benefit  and  investment  figures  in  the  baseline  Table  5  are  discounted  over  a 
period  of  three  to  six  years.  However,  since  the  cases  used  in  this  research  are  free,  open- 
source  cases,  they  did  not  contain  detailed  information  on  how  the  total  discounted 
numbers  were  calculated.  For  example,  the  case  studies  provided  an  investment 

discounted  over  a  number  of  years,  but  they  did  not  identify  when  in  time  the  investment 
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was  made.  The  investment  may  be  assumed  it  have  occurred  at  time  zero,  but  without 
detailed  information,  the  researcher  chose  not  to  adjust  it.  Along  the  same  lines,  the  cost 
savings,  cost  avoidance,  and  productivity  improvements  were  provided  as  an  average 
annual  savings.  Most  SOA  investments  produce  greater  benefits  the  longer  the  systems 
are  used,  so  it  could  be  assumed  that  over  a  period  of  10  years  for  instance,  the  ROI 
would  be  even  greater.  However,  since  the  case  studies  did  not  report  if  the  benefits  were 
immediate,  gradually  grew,  or  gradually  decreased  over  the  time  period,  a  detennination 
could  not  be  made.  Additionally,  the  discount  rates  used  in  these  cases  were  12%,  which 
is  a  common  figure  for  commercial  industries.  The  DoD,  on  the  other  hand,  is  not  a 
revenue-generating  company  and  therefore  does  not  have  competing  investments  that 
would  warrant  such  a  high  discount  rate.  The  DoD  can  use  the  risk-free  U.S.  Department 
of  the  Treasury  rates  as  a  more  accurate  measure  of  discount  rates.  The  daily  treasury 
yield  rates  for  2011  for  three-year  investments  has  fluctuated  between  .5%  and  1.5%  with 
the  average  rate  for  the  first  six  months  of  2011  being  1.05%.  (U.S.  Department  of 
Treasury,  n.d.).  A  direct  comparison  to  the  investments  presented  in  Figure  4  would  be 
unfair  because  the  exact  calculations  conducted  in  the  case  studies  were  not  stated. 
However,  in  general,  using  a  lower,  more  accurate  discount  rate  for  DoD  investments 
would  create  a  much  higher  ROI  when  compared  to  those  realized  in  industry. 

Table  5  also  displays  a  calculated  payback  period.  A  payback  period  is  a  good 
measure  for  determining  how  long  it  will  take  to  recoup  an  initial  investment.  To 
calculate  the  payback  period,  the  annual  cost  savings/annual  cost  avoidance  columns 
were  summed  to  detennine  the  net  cash  flow.  The  net  cash  flow  calculated  represents  a 
periodic  undiscounted  cash  flow.  The  payback  period  was  calculated  using  Equation  2. 

Payback  period  =  (investment/net  cash  flows)  *  12  months  Equation  (2) 
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Table  5.  Calculated  ROI  from  Cost  Savings  and  Cost  Avoidance 


Company 

Reported  ROI 

Calcuated 

ROI  from  Cost 
Savings  /  Cost 
Avoidance 

Average 

Annual  Cost 

Savings 

Average 

Annual  Cost 

Avoidance 

Average 

Annual 

Productivity 

Improvement 

Benefit 

(discounted) 

Investment 

(discounted) 

NPV 

Discount  % 

Discount 

Period 

(Years) 

Payback 

(months) 

Blue  Cross 

Blue  Shield 

332% 

330% 

$2,380,000 

$0 

$90,000 

$14,330,000 

$3,320,000 

$11,010,000 

12% 

6 

16.7 

Mobile 

Telecom 

625% 

136% 

$1,100,000 

$0 

$3,570,000 

$10,120,000 

$1,400,000 

$8,720,000 

12% 

3 

15.3 

Global 

Provider  for 

470% 

-18% 

$0 

$387,853 

$2,827,485 

$8,080,525 

$1,417,846 

$6,662,679 

12% 

3 

43.9 

Services  and 

Fac  Mgmt  Co 

360% 

-100% 

$0 

$0 

$1,140,000 

$2,744,982 

$596,674 

$2,148,309 

12% 

3 

European 

based 

212% 

-18% 

$478,463 

$0 

$1,801,860 

$5,472,842 

$1,753,242 

$3,719,600 

12% 

3 

44.0 

International 

Finance  Firm 

252% 

-31% 

$101,015 

$329,054 

$2,669,439 

$6,627,447 

$1,882,568 

$4,744,879 

12% 

3 

52.5 

Global  Media 
Consulting 

244% 

107% 

$111,609 

$198,140 

$332,626 

$1,541,718 

$447,938 

$1,093,780 

12% 

3 

17.4 

International 

Insurance 

256% 

7% 

$143,839 

$0 

$427,328 

$1,428,180 

$401,607 

$1,026,573 

12% 

3 

33.5 

Healthcare 

Services 

346% 

146% 

$0 

$2,870,000 

$3,720,000 

$15,800,000 

$3,500,000 

$12,300,000 

12% 

3 

14.6 

Global 

Consumer 

265% 

165% 

$270,689 

$0 

$195,366 

$1,118,547 

$306,370 

$812,176 

12% 

3 

13.6 

Average 

336% 

72% 

27.9 

1.  Quantifiable  Benefits 

Table  6  identifies  the  commonalities  of  the  associated  costs  that  were  identified  as 
cost  benefits  from  industry.  These  categories,  or  variations  thereof,  constituted  the 
makeup  of  the  cost  savings,  cost  avoidance,  and  productivity  improvement  figures  in 
Table  5. 
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Table  6.  Quantitative  Benefit  Categories 


Benefit  Categories 

Examples  of  Quantitative 
Measurements 

Benefit  Metrics  Examples 

Cost  Reduction 

Reduced  software  upgrade 
costs,  elimination  of 
hardware  and  associated 
operations  costs,  and 
reduced  personnel 
required. 

Cost  benefits  are  directly 
related  to  decreased 
software/hardware  costs, 
licensing  costs,  or  reduction 
in  full  time  equivalents 
(FTEs). 

Avoidance  from  Future 

Costs 

Decreased  staff,  decreased 
power  consumption,  and 
elimination  of  outsourcing. 

All  costs  can  be  calculated 
based  on  current  rates, 
adjusted  for  inflation. 

Avoidance  of  New 

Investment  Costs 

Purchase  of  new 
infrastructure  or  software. 

Cost  of  replacing  a  modular 
service  compared  with 
replacing  an  entire  system. 

Increase  IT  Staff  Efficiency 

Reduced  repair  time  for 
network  services  and 
security  monitoring. 

Calculate  the  difference 
between  current 
maintenance  costs  and 
maintenance  costs  in  an 

SOA  project. 

Improved  Administrative 
Efficiency/Enhanced  User 
Productivity 

Improved  quality  of  the 
help  desk  and  customer 
satisfaction. 

The  help  desk  knows  of  the 
problem  before  users  call  to 
report,  allowing  them  to 
answer  the  call  quickly. 

Increased  Application 

Availability/Reduced 

Downtime 

Downtime  results  in 
missed  sales,  trading 
opportunities  lost,  and  a 
decrease  in  customer 
satisfaction  and  brand 
equity. 

Downtime  can  be  related  to 
productivity  of  a  user  by  an 
hourly  rate  of  pay.  Sales  can 
be  calculated  per  hour  to 
determine  revenue  lost. 

Software  Reuse 

Less  development  time, 
less  testing  time,  and 
overall  lower  project  costs. 

Actual  cost  comparison  of 
reused  software  to  newly 
developed  software. 

Training  costs  and 
productivity  loss  of  users 
learning  a  new  system. 

Simplified  User  Interface 

Decreased  user  learning 
time. 

Reduced  training  costs  and 
increased  productivity. 
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2.  Nonquantifiable  Benefits 

In  addition  to  monetary  cost  savings,  the  case  studies  listed  several  benefits  that 
were  not  monetized  or  that  the  researcher  removed  because  they  did  not  correspond  well 
to  any  of  the  three  financial  management  characteristics  of  cost  savings,  cost  avoidance, 
and  productivity  improvement.  Table  7  lists  these  categories  as  well  as  briefly  describes 
how  they  may  impact  the  DoD. 


Table  7.  Qualitative  Benefit  Categories 


Benefit  Categories 

Examples  of  Qualitative 
Measurements 

Relationship  to  the  DoD 

Business  Staff  Efficiency 

Information  delivered  to 
managers  more  quickly  and 
accurately  improves 
decision-making. 

Delivering  timely  and 
accurate  information  is  vital 
to  military  leaders. 

Business  Credibility 

Equates  to  more  business 
because  other  companies 
view  their  system  as 
available  and  reliable. 

Availability  and  reliability 
of  systems  in  the  DoD  is  a 
productivity  improvement. 

Reduced  Duplication  of 
Effort 

Information  is  entered  once 
and  available  to  all  users. 
(This  could  be  a 
productivity  improvement 
as  well  but  was  listed 
separately  as  a  qualitative 
benefit.) 

Ensures  accuracy  and 
consistency  of  data.  It  also 
saves  time  inputting  data  or 
fixing  mismatched  data. 

Faster  Time  to  Market 

Difference  in  the  amount 
of  time  a  product  is 
available  compared  to  the 
current  time  to  market. 

Faster  delivery  of  vital 
intelligence  or  logistics 
when  and  where  required. 

Scalability 

The  ability  to  increase  size 
or  volume  without 
degradation. 

The  ability  of  the  service  to 
be  scaled  in  accordance 
with  the  changing  mission. 

Flexibility 

Flexibility  is  achieved 
through  increased  agility 
and  the  potential  for  reuse. 

Flexibility  allows  the 
system  the  ability  to  quickly 
adapt  to  changing  needs  or 
requirements. 
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A  case  study  for  one  company  that  was  not  included  in  the  ROI  calculations 
because  an  ROI  was  not  provided,  nor  could  it  be  calculated,  was  the  one  for  the  United 
States  Military  Entrance  Processing  Command  (USMEPCOM).  This  case  study  is  worth 
mentioning,  however,  because  it  deals  directly  with  the  DoD  and  because  the 
USMEPCOM  was  so  successful  in  its  implementation  of  an  SOA  system  that  it  was  able 
to  decrease  the  costs  of  a  new  project  by  $56  million,  which  won  the  award  for  Best 
Return  on  Investment  in  the  BPM  Case  Study  Competition  conducted  by  Object 
Management  Group  and  BPTrends.  One  particular  aspect  of  USMEPCOM‘s  success  was 
reusability.  USMEPCOM  was  able  to  put  reusability  to  work  and  complete  a  security 
project  originally  estimated  at  six  months  in  only  two  weeks  (Network  Centric 
Operations  Industry  Consortium  [NCOIC],  2010). 

C.  PUBLISHED  SURVEYS 

In  addition  to  specific  case  studies,  the  researcher  analyzed  published  surveys  in 
order  to  identify  whether  the  data  from  the  case  studies  were  representative,  to  understand 
the  perspective  industry  has  on  SOA,  and  to  discover  some  best  practices  in  industry. 
Finally,  these  perceptions  and  best  practices  were  compared  to  the  case  studies  to 
detennine  what,  if  any,  of  the  concepts  materialized. 

The  first  aspect  analyzed  was  to  determine  what  industry  perceived  as  value  for 
its  IT  investments.  In  January  2008,  Aberdeen  Group  published  a  report  after  surveying 
4,600  business  and  IT  decision  makers.  The  question  the  survey  asked  was  what  role 
participants  thought  IT  would  play  in  their  businesses  in  the  current  year.  The  results  of 
the  survey  are  shown  in  Figure  6. 
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Improves  Operational  Efficiency 


39% 


%  of  respondents 

Figure  6.  Primary  Roles  of  Business  Technologies  in  2008 

(From  Dortch,  2008) 

Of  the  six  most-often  cited  categories  in  this  survey,  five  of  them  were 
experienced  by  companies  in  the  researcher4 s  selected  case  studies.  Only  one,  improves 
communication,  was  not  cited  as  a  benefit. 

Another  survey  of  North  American  and  European  companies  cited  improved 
customer  service  and  faster  time  to  market  as  the  largest  benefit  participants  expected 
from  their  IT  investments.  These  benefits  were  also  identified  as  benefits  in  the  case 
studies.  However,  when  these  same  companies  were  asked  what  the  primary  driver  of  the 
SOA  vision  within  their  organization  was,  IT  cost  savings  was  the  most  frequent  answer, 
with  30%  of  respondents  citing  that  reason,  followed  by  customer  service  improvement 
and  faster  time  to  market  at  23%  and  21%,  respectively  (Ritter  &  Evans,  n.d.).  Aberdeen 
Group  (2008)  conducted  a  study  of  the  SOA  efforts  of  400  companies,  and  among  the 
companies  identified  as  best-in-class,  62%  reported  improved  business  agility  as  their 
primary  driver  for  SOA  deployment.  Reducing  operating  costs  tied  for  third  at  39% 
(Dortch,  2008).  Forrester  Research  claimed  81%  to  84%  of  SOA  users  identified  the 
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drivers  for  SOA  as  improving  business  and  application  flexibility,  while  70%  to  75%  of 
SOA  users  responded  that  lowering  business  and  application  costs  were  the  drivers  for 
SOA  (Heffner  &  Fulton,  2008). 


IBM  conducted  in-depth  interviews  with  actual  members  of  the  project  teams 
from  35  SOA  implementations  worldwide,  spanning  1 1  industries.  The  benefits  reported 
in  these  interviews  are  shown  in  Figure  7. 


Benefits  reported  hy  the  SOA  projects  studied. 


Improve  flexibility 
Decrease  cost 
Reduce  risk 
Increase  revenue 
Enable  new  products 
Enable  compliance 


Percent 
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26 


Figure  7.  Benefits  Reported  by  the  SOA  Projects  Studied  by  IBM 

(From  DiMare,  2009) 


All  but  one  company  in  the  IBM  study  reported  a  decrease  in  costs  as  a  benefit 
from  SOA  implementation  (DiMare,  2009).  In  addition,  in  a  study  of  over  900  IT  and 
business  decision  makers,  over  60%  who  reported  reducing  cost  as  a  major  objective  of 
SOA  are  currently  meeting  or  exceeding  their  cost  reduction  objectives  (IBM  Global 
Technology  Services,  2009). 

The  data  described  in  Figures  6  and  7,  as  well  as  published  surveys  previously 
mentioned,  support  the  fact  that  reducing  costs  is  an  important  factor  in  industry,  and 
most  companies  have  been  successful  at  achieving  their  cost-reduction  goals.  A  report 
published  in  2009  concluded  that  only  6%  of  organizations  surveyed  after  adopting  an 
SOA  had  a  negative  ROI.  Of  the  remainder,  57%  broke  even  and  37%  experienced  a 
positive  ROI  (Computer  Economics,  2009).  Furthermore,  in  a  separate  study, 
approximately  50%  reported  that  their  SOA  investment  had  at  least  paid  for  itself.  This 
seems  to  be  representative  of  the  findings  in  the  case  studies,  as  six  of  the  10  selected 
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cases  reported  a  positive  ROI.  However,  a  positive  ROI  of  cost  savings  is  not  a  foregone 
conclusion.  In  fact,  cost  reduction  by  itself  does  not  encompass  all  the  benefits  offered  by 
an  SOA  implementation.  Furthermore,  it  is  not  always  the  primary  driver  in 
implementing  SOA  for  industry,  as  has  been  seen  in  several  published  surveys.  Many 
other  factors  play  into  the  decision. 

The  next  question  is,  If  these  are  the  reported  outcomes,  then  is  SOA  still 
prevalent  in  industry?  The  researcher  collected  data  to  determine  whether  SOA 
implementation  is  on  the  rise,  holding  steady,  or  declining  in  industry.  The  results  from 
the  data  indicate  that  SOA  implementation  is  on  the  rise.  One  report  showed  that  the 
percentage  of  organizations  making  the  transition  to  a  service-oriented  model  jumped 
from  18%  in  2006  to  58%  in  2008  (Computer  Economics,  2009).  In  addition,  in  2008, 
70%  of  SOA  users  said  they  planned  to  increase  their  use  of  SOA,  while  only  3% 
planned  to  decrease  their  use  (Heffner  &  Fulton,  2008). 

Two  primary  benefits  of  SOA  adoption  are  decreased  risk  and  reusability.  Risk 
mitigation  encompasses  many  factors  listed  in  the  cases  and  surveys.  These  include 
flexibility  that  allows  IT  to  more  quickly  react  to  changing  demands,  scalability  to 
increase  scope  as  needed,  and  reusability  that  implements  proven  technologies  rather  than 
attempting  to  develop  a  service  from  scratch.  In  addition,  proven  technologies  increase 
the  availability  and  stability  factors  of  a  system  because  they  have  already  been  tested 
and  implemented  previously.  Risk  mitigation  is  extremely  important  in  the  DoD  because 
all  too  often,  systems  are  delivered  late,  over  budget,  and  without  the  capability  to 
perfonn  as  they  were  meant  to. 

As  mentioned,  an  important  quality  of  risk  mitigation  is  reusability.  Reusability  is 
often  considered  as  a  necessary  component  to  making  SOA  cost  effective.  One  reason  for 
this,  as  stated  by  DiMare  of  the  IBM  Institute  for  Business  Value,  is  increased  reuse 
leads  to  reduced  maintenance,  which  leads  to  decreased  costs;  or  in  another  path, 
increased  reuse  leads  to  reduced  integration  time,  which  leads  to  reduced  integration  cost 
and  thus  to  decreased  costs”  (2008,  p.  7).  The  true  value  of  reuse  is  in  the  standardization 
of  business  processes  (IBM,  2005).  One  survey  concluded  that  90%  of  organizations  see 

reuse  as  a  critical  metric  for  success  (Ritter  &  Evans,  n.d.).  Poulin  and  Hinder  (2006) 
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suggested  that  the  cost  of  reusing  an  SOA  component  is  about  half  the  cost  of  reusing 
traditional  components.  Forrester  Research  reports  that  SOA  development  can  be  almost 
twice  the  cost  of  traditional  component  development,  but  once  the  component  is  reused 
over  and  over,  SOA  becomes  30%  more  cost  effective  (Kobielus,  2005).  As  an  example, 
Delta  reported  significant  cost  savings  when  reusing  components  (HP,  2010). 
Furthennore,  AT&T  claimed  reuse  of  a  single  service  saved  it  between  50%  to  85%  of 
the  cost  of  building  custom  interfaces  (Erickson,  2006). 

In  conclusion,  the  reported  surveys  show  that  industry  believes  cost  is  an 
important  facet  of  an  SOA  implementation  and  that  industry  would  not  move  to  an  SOA 
if  it  did  not  provide  some  type  of  positive  ROI.  However,  a  straight-line  cost  reduction 
was  typically  not  the  objective  of  industry  when  implementing  an  SOA.  Instead,  industry 
focused  primarily  on  efficiencies  and  providing  a  flexible  business  position.  The 
objectives  of  an  SOA  implementation  and  the  actual  benefits  realized  that  were  identified 
in  the  surveys  closely  resembled  those  in  the  analyzed  case  studies.  This  allowed  the 
researcher  to  conclude  that  the  case  studies  provided  an  accurate  representation  of 
industry  and  that  they  can  be  used  to  arrive  at  an  industry  benchmark.  Furthermore,  the 
surveys,  along  with  the  case  studies,  formed  the  basis  for  the  researcher‘s  conclusion  of 
industry  best  practices. 

D.  INDUSTRY  BEST  PRACTICES 

The  research  provided  two  examples  of  best  practices.  These  include  ensuring  that 
flexibility  is  built  into  the  implementation  and  using  an  incremental  approach.  As 
evidenced  by  the  surveys  and  case  studies,  flexibility  was  at  or  near  the  top  of  the  list  of 
objectives  when  implementing  SOA.  In  addition,  it  was  often  recognized  as  a  valued 
benefit  as  a  result  of  implementing  SOA.  The  ability  to  react  and  change  course  in  a 
rapidly  changing  environment  was  considered  an  enormous  benefit.  In  turn,  any  SOA 
project  the  DoD  intends  to  implement  must  ensure  flexibility.  It  is  not  only  the  business 
environment  that  is  changing  rapidly  but  also  the  military  environment  in  terms  of  the 
threats  faced  by  the  various  Services.  No  longer  are  mass  armies  attacking  one  another. 
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The  face  of  warfare  has  become  terrorist  groups  who  continue  to  adapt  their  tactics.  The 
DoD  acquisition  strategy  must  be  able  to  adapt  and  react  to  these  changes,  and  flexibility 
is  the  key. 

The  second  best  practice  drawn  from  the  research  is  use  of  an  incremental 
approach  in  implementing  an  SOA.  First,  it  is  very  difficult  to  gather  the  resources  to 
make  an  enterprise-wide  conversion  from  legacy  systems  to  SOA.  A  better  practice  for 
companies  is  to  adopt  SOA  on  an  opportunistic  basis  such  as  when  legacy  system 
integration  is  required  (Computer  Economics,  2009).  In  the  same  way,  the  DoD  should 
start  small  with  near-term  or  easily  implemented  requirements,  and  build  from  there. 
Furthennore,  initially  attack  the  low-hanging  fruit  by  introducing  SOA  services  that  will 
provide  an  immediate  bang  for  the  buck.  When  analyzing  the  best  practices  from  all  the 
case  studies,  one  thing  nearly  all  had  in  common  was  that  they  introduced  a  specific 
service  to  solve  a  specific  problem.  They  did  not  attempt  a  massive  replacement  of  all 
their  systems  at  once,  but  instead  focused  on  specific  areas  they  felt  needed  improvement 
and  implemented  a  solution  in  that  area.  In  addition  to  mitigating  risk  and  being  less 
expensive,  this  approach  allows  an  organization  to  learn  from  the  early  implementations, 
thereby  reducing  the  learning  curve  for  future  implementations. 

E.  IMPLICATIONS  OF  RESEARCH  FOR  THE  DOD 

As  discussed  in  the  literature  review,  SOA  and  OA  use  many  of  the  same 
concepts,  which  allowed  them  to  be  treated  as  similar  in  the  framework  of  this  thesis. 
Many  of  the  objectives  identified  in  the  surveys  were  also  identified  in  the  analyzed  case 
studies.  Furthermore,  many  of  the  outcomes  reported  in  the  surveys  also  closely  matched 
the  realized  benefits  found  in  the  case  studies.  This  means  the  DoD  can  expect  similar 
outcomes  to  those  achieved  by  industry.  In  addition,  the  DoD  can  learn  from  the  industry 
best  practices  identified  in  this  thesis  and  use  that  information  in  its  own  implementation 
of  OA. 

This  research  serves  as  a  benchmark  measure  of  what  ROI  the  DoD  can  expect  if 
it  implements  an  OA.  The  baseline  ROIs  reported  in  Table  4  were  rather  high  and  offered 
a  very  quick  payback  period.  In  addition,  when  the  ROI  was  calculated  solely  from  cost 
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savings,  as  shown  in  Table  5,  the  results  were  still  respectable.  This  was  encouraging 
since  many  companies  were  not  focused  on  cost  savings,  but  on  other  areas  such  as  the 
flexibility  to  position  their  company  competitively  for  the  future.  The  DoD  can  benefit 
from  this  research  in  its  acquisition  of  systems.  The  three  primary  areas  of  interest  in 
DoD  acquisitions  are  cost,  schedule,  and  performance.  Although  cost  was  the  focus  of 
this  thesis,  schedule  and  perfonnance  were  found  to  be  very  much  impacted  by  SOA  in 
industry.  For  example,  ensuring  a  flexible  system  has  a  direct  impact  on  schedule.  The 
reason  companies  desire  a  flexible  system  is  so  they  can  shift  gears  quickly  to  take 
advantage  of  a  changing  environment.  Although  schedule  impact  may  not  be  seen  in  the 
initial  investment,  it  becomes  evident  in  subsequent  investments.  There  are  several  causes 
of  this,  such  as  the  reusability  factor,  which  allowed  USMEPCOM  to  decrease  the 
schedule  time  of  a  follow-on  project  from  six  months  to  two  weeks.  In  addition  to 
improving  schedule,  increased  perfonnance  was  a  benefit  seen  in  the  case  studies.  Often 
listed  as  staff  efficiencies,  workers  were  able  to  spend  less  time  on  issues  such  as 
maintenance  and  instead  focus  on  other  areas  that  would  benefit  the  company.  The 
schedule  and  perfonnance  aspects  of  SOA  may  be  equally,  if  not  more,  beneficial  than 
the  potential  cost  savings.  This  is  because  just  over  half  of  the  companies  included  in 
Table  5  experienced  a  positive  ROI  from  cost  savings  and  cost  avoidance  alone,  but  all  of 
the  companies  analyzed  experienced  some  sort  of  efficiency  that  they  concluded  had 
resulted  in  an  overall  positive  ROI. 

DoD  acquisitions  would  also  benefit  from  the  risk  mitigation  offered  by  SOA 
projects.  Some  of  the  best  practices  learned  from  this  research  include  reusability  of 
technologies,  using  an  incremental  approach,  and  building  the  system  with  a  high  level  of 
flexibility  and  scalability,  all  of  which  equate  to  reduced  risk.  Because  many  acquisitions 
programs  fail  to  meet  their  cost,  schedule,  and  performance  goals,  implementing  a 
methodology  that  reduces  the  associated  risks  would  seem  highly  desirable.  This  thesis 
demonstrated  the  importance  of  flexibility  in  a  system.  With  a  stove-piped  architecture, 
there  is  very  little  flexibility.  Not  only  is  it  inflexible  during  its  useful  life,  but  it  is 
already  inflexible  at  its  inception.  While  in  the  development  stage,  the  program  may  have 
already  changed  due  to  factors  such  as  increased  scope,  technology  obsolescence,  and  so 
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forth.  Even  though  the  acquisition  community  requires  a  risk  mitigation  strategy  for  its 
projects,  it  is  different  than  the  risk  mitigation  offered  by  OA.  Often,  risk  mitigation 
strategies  for  stove-piped  systems  are  implemented  early  on.  This  would  mean  that  the 
features  and  requirements  of  a  system  would  be  decided  early  on  in  a  program ‘s 
development  and  would  remain  unchanged  throughout  the  implementation  phase. 
However,  it  is  likely  that  requirements  will  change  throughout  the  implementation 
because  needs  and  technologies  change,  knowledge  is  incomplete  at  the  start,  or  a  series 
of  other  reasons  exists  (Campbell,  2010).  In  fact,  locking  in  requirements  too  early  in  the 
process  may  lead  to  inflexibility  in  the  program  (Patterson,  Ott,  &  Giglio,  2009), 
resulting  in  the  program  that  does  not  achieve  all  its  goals.  One  the  other  hand,  OA  offers 
the  flexibility  to  adjust  to  this  changing  environment. 

As  a  way  ahead  for  DoD,  it  is  imperative  to  develop  a  method  of  measuring  the 
actual  value  of  its  investments,  ensuring  flexibility  in  its  systems,  as  well  as 
implementing  risk  mitigation  strategies.  Although  there  are  several  ways  of 
accomplishing  this,  one  study  has  already  proposed  a  method  to  solve  these  issues  and 
could  be  used  as  a  model  going  forward.  The  Naval  Postgraduate  School  along  with 
PEO-IWS  conducted  a  pilot  study  to  apply  Knowledge  Value  Added  +  Real  Options  + 
Integrated  Risk  Management  +  Portfolio  Optimization  (KVA  +  RO  +  IRM  +  PO)  to 
estimate  the  value  created  by  inserting  capabilities  into  the  Aegis  Weapons  Systems 
(AWS)  through  the  Advanced  Capability  Build  process  (Mun,  Housel,  &  Wessman, 
2010).  The  study  looked  at  the  23  capabilities  to  be  inserted  into  the  AWS  while 
considering  issues  such  as  value  to  the  warfighter,  risks,  and  a  constrained  budget.  Using 
this  toolset,  the  researchers  were  quickly  able  to  estimate  the  effects  of  varying  capability 
insertions.  In  addition,  the  researchers  were  able  to  quickly  change  the  parameters  such  as 
adding  new  capabilities  or  additional  risk  factors.  This  provided  a  great  deal  of  flexibility 
to  the  decision  maker.  Although  not  every  system  would  require  such  an  in-depth 
analysis,  using  a  model  such  as  this  could  be  applied  to  most  any  investment  and  provide 
the  ability  to  better  manage  acquisition  projects. 

One  concept  that  was  proven  successful  in  the  AWS  study  was  the  use  of  KVA. 
The  DoD  could  consider  using  KVA  to  measure  the  value  of  a  project  rather  than  ROI, 
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because  it  uses  a  derived  value  for  the  numerator.  This  would  ensure  benefits  would  be 
analyzed  in  objective,  common  units  and  will  provide  a  more  accurate  measure  of  value. 
This  is  important  because  Mun,  Housel,  and  Wessman  (2010)  found  little  correlation 
between  actual  cost  of  insertions  and  their  military  value  when  studying  the  AWS.  In 
addition,  the  DoD  should  implement  RO  into  its  acquisition  of  OA  systems.  RO  takes 
into  consideration  that  projects  have  some  amount  of  uncertainty  and  provides  the 
decision  maker  flexibility  to  exercise  or  abandon  those  options  at  different  points  in  time 
when  more  information  is  known  or  the  requirements  change  (Mun  &  Housel,  2006.)  The 
use  of  RO  would  address  the  industry  best  practice  of  flexibility  by  allowing  decisions  to 
be  made  when  more  complete  information  is  available.  Furthennore,  RO  adheres  to  using 
an  incremental  approach,  another  industry  best  practice,  by  allowing  for  phased  options 
and  the  option  to  wait  or  defer  additional  investments.  Finally,  since  RO  allows  the 
decision  maker  to  assess  the  project  at  various  points,  it  can  be  used  to  frame  strategies  to 
reduce  risk. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSION 

The  main  purpose  of  this  thesis  was  to  analyze  cost  savings  from  various 
industries  following  the  implementation  of  an  SOA.  The  research  had  several  objectives. 
The  first  objective  was  to  establish  a  benchmark  of  perfonnance  outcomes,  focusing  on 
cost  savings  experienced  in  industry  in  order  to  determine  the  benefits  the  government,  or 
the  DoD  specifically,  could  expect  to  realize  in  its  push  to  transition  to  a  more  open 
architecture  model.  The  second  objective  was  to  detennine  some  industry  best  practices 
that  may  be  used  by  the  DoD  in  this  process. 

The  DoD  can  benefit  from  the  acquisition  of  an  OA.  The  DoD  is  ready  to 
implement  OA,  and  the  research  findings  in  this  thesis  show  that  implementation  of  OA 
in  the  DoD  would  work.  Guidelines  outlining  the  acquisition  of  OA  are  already  in  place. 
In  addition,  guidelines  for  the  use  of  MOSA  and  COTS  are  already  published.  Very  little 
would  need  to  change.  However,  the  DoD  cannot  assume  OA  will  solve  all  of  its  IT 
system  needs,  nor  can  it  assume  that  OA  will  save  the  DoD  a  great  deal  of  money.  As 
seen  in  the  cases  and  best  practices  from  industry,  the  DoD  must  be  cautious  as  to  which 
projects  it  pursues  by  focusing  on  solving  very  specific  issues.  The  DoD  cannot  look  for 
the  one-size-fits-all  approach,  but  rather  identify  those  projects  that  will  provide  the  most 
bang  for  the  buck  and  pursue  those  using  OA  systems. 

SOA  in  industry  and  OA  in  the  DoD  can  be  considered  similar  concepts,  and 
therefore  the  results  seen  from  an  SOA  implementation  in  industry  can  be  expected  by 
the  DoD.  The  industry  cost-savings  ROI  calculation  was  72%,  and  while  this  may  seem 
attractive,  many  other  factors  must  be  weighed  by  the  DoD  before  implementing  an  SOA 
project.  The  ROI  is  sensitive  to  many  aspects,  and  there  is  no  guarantee  the  outcome  will 
always  be  positive.  In  fact,  only  six  of  the  ten  case  studies  analyzed  reported  a  positive 
ROI  based  on  cost  savings  and  cost  avoidance  alone.  Therefore,  actual  ROI  from  cost 
savings  could  vary  greatly.  However,  other  benefits,  including  productivity 
improvements  and  non-quantifiable  benefits,  can  more  than  make  up  the  difference.  The 
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focus  for  the  DoD,  then,  should  not  be  solely  on  the  cost  savings  SOA  can  provide,  but 
on  benefits  such  as  flexibility  and  scalability  that  will  allow  for  improvements  in  the  long 
tenn.  Furthermore,  the  DoD  should  assess  the  reusability  factor  of  services  when  making 
plans  to  implement  OA.  Not  only  does  reusability  save  money,  but  it  also  decreases 
project  risk.  Finally,  immediate  mass  implementation  of  SOA  is  not  recommended. 
Instead,  the  DoD  should  take  an  incremental  approach,  focusing  on  particular  needs  and 
requirements,  and  implement  SOA  where  it  will  have  the  greatest  impact  first. 

B.  RECOMMENDATIONS  FOR  DOD 

Based  on  the  conclusions,  the  researcher  has  developed  several  recommendations 
that  will  benefit  the  DoD  as  it  continues  its  push  toward  open  systems.  The  first 
recommendation  is  to  focus  not  only  on  cost  savings,  but  on  overall  value  offered  by  an 
open  system.  Many  benefits  such  as  flexibility,  scalability,  and  reusability  will  position 
DoD  to  rapidly  adjust  their  systems  to  the  changing  combat  mission  and  environment  as 
well  as  reduce  future  risk.  Flexibility,  specifically,  was  noted  as  an  industry  best  practice, 
and  the  DoD  should  incorporate  system  flexibility  to  the  greatest  extent  possible. 
Although  these  benefits  may  equate  to  cost  savings  in  the  future,  they  are  not  included  in 
the  current  cost  savings  calculations.  Furthermore,  making  decisions  solely  on  cost 
savings  sends  the  message  that  the  DoD  is  only  concerned  with  reducing  costs.  This 
essentially  means  the  goal  is  to  reduce  costs  to  zero,  which  is  a  fallacy  in  logic  as 
addressed  in  Chapter  IV.  In  order  to  make  a  completely  informed  decision,  the  DoD 
should  consider  reducing  the  weight  given  to  ROI  as  a  result  of  cost  savings  in  its 
decision-making  process  and  instead  attempt  to  incorporate  all  benefits  associated. 

The  second  recommendation  for  the  DoD  is  to  take  an  incremental  approach  and 
implement  OA  where  results  will  be  immediate.  SOA  is  not  a  one-size-fits-all  solution. 
Therefore,  mass  implementation  of  SOA  is  not  recommended.  Instead,  the  DoD  should 
assess  its  current  architecture  and  focus  its  efforts  on  particular  needs  and  requirements. 
SOA  should  then  be  implemented  where  it  will  have  the  greatest  impact  first.  The  DoD 
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should  start  small  with  near-term  or  easily  implemented  requirements,  initially  attacking 
the  low-hanging  fruit  by  introducing  SOA  services  that  will  provide  the  most  bang  for  the 
buck. 


C.  RESEARCH  SHORTCOMINGS 

The  primary  shortcoming  of  this  thesis  is  the  validity  of  the  data.  Because  the  data 
was  freely  available,  it  was  provided  more  as  a  marketing  tool  than  as  a  qualitative 
representation  of  the  typical  outcome  in  industry.  In  this  way,  the  companies  sponsoring 
the  case  studies  on  the  implementation  of  one  of  their  SOA  services  most  likely  would 
not  report  a  failed  SOA  implementation,  but  would  only  report  the  projects  that 
succeeded.  In  addition,  detailed  information  on  how  the  research  companies  conducted 
their  calculations  was  not  available.  This  prohibited  accurate  calculations  by  the 
researcher  because  it  could  not  be  detennined  when  in  time  investments  were  made  or 
when  benefits  were  realized.  More  detailed  infonnation  is  available  by  purchasing  the 
complete  studies,  but  this  was  beyond  the  scope  and  economic  feasibility  of  this  thesis. 

As  an  additional  shortcoming,  the  ROI  relating  to  cost  savings  and  cost  avoidance 
was  calculated  on  10  case  studies  that  reported  benefits  in  separate  cost  categories.  In  that 
respect,  10  case  studies  are  not  enough  to  be  considered  representative  of  results  of  SOA 
implementation  by  industry  in  general.  Also,  many  of  the  surveys  listed  cannot  be 
considered  representative.  Several  surveys  provided  a  disclaimer  stating  that  the  results 
should  not  be  considered  representative  of  all  SOA  implementations.  Also,  surveys  by 
their  very  nature  are  somewhat  biased  because  typically  only  those  respondents  with  a 
vested  interest  actually  complete  the  surveys. 

D.  RECOMMENDATIONS  FOR  FUTURE  STUDIES 

While  conducting  the  research  and  writing  for  this  thesis,  the  researcher  identified 
several  issues  that  could  be  developed  and  addressed  in  the  future. 

1.  Research  Complete  Reports 

The  primary  shortcoming  of  this  thesis  was  the  lack  of  access  to  complete 
company  reports.  Currently,  detailed  reports  are  only  offered  for  a  fee.  If  possible,  future 
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studies  should  assess  detailed  reports  to  understand  the  underlying  meaning  of  what  SOA 
is  actually  bringing  to  the  company  rather  than  rely  on  a  brief  synopsis  whose  primary 
use  is  as  a  marketing  tool.  This  may  include  looking  at  several  individual  companies  in 
great  detail  to  better  assess  their  success  or  failure  in  implementing  SOA.  Additionally, 
this  may  provide  a  more  accurate  depiction  of  when  in  time  benefits  were  realized. 
Furthennore,  non-profit  organizations  should  be  analyzed  as  they,  similar  to  the  DoD, 
lack  the  goal  of  revenue  generation  and  therefore  might  be  more  representative  of  the 
results  the  DoD  would  experience. 

2.  Analyze  Actual  OA  Implementations  in  the  DoD 

Analyzing  actual  DoD  implementations  on  their  successes  or  failures  should  be  a 
focus  of  future  research.  However,  the  research  should  be  conducted  on  the  basis  of 
overall  value  and  not  just  on  cost  savings  to  detennine  the  true  value  of  the  project.  In 
that  way,  the  research  should  not  focus  solely  on  ROI  from  cost  savings  or  cost 
avoidance,  but  should  also  make  calculations  for  productivity  improvements  as  well  as 
the  increased  flexibility  and  scalability  provided. 

3.  Assess  the  Viability  of  Reusability  in  the  DoD 

This  thesis  discussed  the  importance  of  reusability  and  the  benefits  it  can  provide 
in  industry,  but  it  is  unclear  whether  these  benefits  can  translate  into  the  DoD  due  to 
restrictions  on  testing  and  security  of  the  software.  One  of  the  primary  benefits  reusability 
offers  is  a  proven  technology  that  can  be  reused  in  a  slightly  different  capacity  over  and 
over  again.  However,  if  testing  and  security  restrictions  apply  to  reused  software,  it  is 
unclear  whether  the  DoD  will  benefit  from  the  reusability  factor.  One  recommendation 
for  future  research  is  to  look  at  reused  software  in  DoD  that  did  go  through  the  various 
testing  and  security  checks  and  assess  whether  it  was  necessary.  For  example,  new 
research  could  look  at  whether  reliability  was  diminished  or  whether  security  was 
reduced  in  the  reused  software. 
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